[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:28 dist-test-slave-794d krb5kdc[9672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:28 dist-test-slave-794d krb5kdc[9672](info): set up 2 sockets
Sep 25 22:00:28 dist-test-slave-794d krb5kdc[9672](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 22:00:29.748080 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:29 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:29.869234 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40933
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:29.952121 9685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:29.952389 9685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:29.952414 9685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:29.953660 9685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:29.953701 9685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:29.953715 9685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:29.953727 9685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:29.953737 9685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:29.955309 9685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:29.955557 9685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:29.955750 9685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:29.961344 9693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:29.961344 9690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:29.961401 9685 server_base.cc:1034] running on GCE node
W20260925 22:00:29.961401 9691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:29.961978 9685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:29.962287 9685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:29.963452 9685 hybrid_clock.cc:648] HybridClock initialized: now 1790373629963435 us; error 38 us; skew 500 ppm
Sep 25 22:00:29 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373629, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:29.966070 9685 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:29.966583 9685 webserver.cc:460] Webserver started at http://127.9.112.126:39203/ using document root <none> and password file <none>
I20260925 22:00:29.966821 9685 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:29.966877 9685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:29.967007 9685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:29.968082 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "da6956c24d23403c8c5f15cb8c76bbfb"
format_stamp: "Formatted at 2026-09-25 22:00:29 on dist-test-slave-794d"
server_key: "4dc86ec94108dbf61cbafd0610b703e0"
server_key_iv: "0516163083a87e1f49208af3bf9180b0"
server_key_version: "encryptionkey@0"
I20260925 22:00:29.968480 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "da6956c24d23403c8c5f15cb8c76bbfb"
format_stamp: "Formatted at 2026-09-25 22:00:29 on dist-test-slave-794d"
server_key: "4dc86ec94108dbf61cbafd0610b703e0"
server_key_iv: "0516163083a87e1f49208af3bf9180b0"
server_key_version: "encryptionkey@0"
I20260925 22:00:29.969801 9685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:29.970575 9700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:29.970796 9685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:29.970891 9685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "da6956c24d23403c8c5f15cb8c76bbfb"
format_stamp: "Formatted at 2026-09-25 22:00:29 on dist-test-slave-794d"
server_key: "4dc86ec94108dbf61cbafd0610b703e0"
server_key_iv: "0516163083a87e1f49208af3bf9180b0"
server_key_version: "encryptionkey@0"
I20260925 22:00:29.970961 9685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:29.990913 9685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:29.993322 9685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:29.993470 9685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:29.997602 9685 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40933
I20260925 22:00:29.997665 9752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40933 every 8 connection(s)
I20260925 22:00:29.998153 9685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:29.998803 9753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:30.001155 9753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb: Bootstrap starting.
I20260925 22:00:30.001734 9753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:30.001993 9753 log.cc:824] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:30.002674 9753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb: No bootstrap required, opened a new log
I20260925 22:00:30.004288 9753 raft_consensus.cc:348] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40933 } }
I20260925 22:00:30.004516 9753 raft_consensus.cc:374] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:30.004555 9753 raft_consensus.cc:729] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6956c24d23403c8c5f15cb8c76bbfb, State: Initialized, Role: FOLLOWER
I20260925 22:00:30.004590 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 9685
I20260925 22:00:30.004689 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:30.004709 9753 consensus_queue.cc:260] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40933 } }
I20260925 22:00:30.004779 9753 raft_consensus.cc:388] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:30.004808 9753 raft_consensus.cc:482] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:30.004859 9753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:30.005548 9665 external_mini_cluster.cc:1428] Setting key 67e244e36b22f1dc3690d72c3a9d29ca
I20260925 22:00:30.006129 9753 raft_consensus.cc:504] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40933 } }
I20260925 22:00:30.006232 9753 leader_election.cc:302] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da6956c24d23403c8c5f15cb8c76bbfb; no voters:
I20260925 22:00:30.006410 9753 leader_election.cc:288] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:30.006589 9757 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40933 } } }
I20260925 22:00:30.006654 9757 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:00:30.006668 9753 sys_catalog.cc:564] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:30.006747 9756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:30.006884 9756 raft_consensus.cc:686] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [term 1 LEADER]: Becoming Leader. State: Replica: da6956c24d23403c8c5f15cb8c76bbfb, State: Running, Role: LEADER
I20260925 22:00:30.007033 9756 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40933 } }
I20260925 22:00:30.007421 9757 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader da6956c24d23403c8c5f15cb8c76bbfb. Latest consensus state: current_term: 1 leader_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6956c24d23403c8c5f15cb8c76bbfb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40933 } } }
I20260925 22:00:30.007484 9757 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:30.007665 9766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:30.008247 9766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:30.009688 9766 catalog_manager.cc:1347] Generated new cluster ID: 87ad9c222405424881e6e76ec8ffe569
I20260925 22:00:30.009737 9766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:30.024804 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.006520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.9.112.126:40933)
0925 22:00:30.006852 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:30.006862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:30.007203 (+ 341us) server_negotiation.cc:407] Connection header received
0925 22:00:30.009853 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:30.009861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:30.009916 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:30.010046 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:30.011921 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.013399 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.014861 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.015638 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.015779 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.020452 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:30.020456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:30.020457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:30.020472 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:30.022571 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.023121 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.023124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.023125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.023243 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.023523 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.023524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.023524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.023850 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:30.023993 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:30.024318 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 22:00:30.024535 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":83,"threads_started":1}
I20260925 22:00:30.034607 9766 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:30.035377 9766 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:30.045432 9766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da6956c24d23403c8c5f15cb8c76bbfb: Generated new TSK 0
I20260925 22:00:30.045666 9766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:30.080508 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40933
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:30.166042 9778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:30.166280 9778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:30.166314 9778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:30.167595 9778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:30.167644 9778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:30.167690 9778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:30.169203 9778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:30.169454 9778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:30.169688 9778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:30.174959 9783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:30.174913 9786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:30.174913 9784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:30.175148 9778 server_base.cc:1034] running on GCE node
I20260925 22:00:30.175462 9778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:30.175696 9778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:30.176896 9778 hybrid_clock.cc:648] HybridClock initialized: now 1790373630176879 us; error 32 us; skew 500 ppm
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:30.179312 9778 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:30.179725 9778 webserver.cc:460] Webserver started at http://127.9.112.65:37627/ using document root <none> and password file <none>
I20260925 22:00:30.179917 9778 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:30.179955 9778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:30.180049 9778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:30.181551 9778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "67acb2bd14c0495c87afccd6c208a716"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "298455520c4566c71979cffb3df02a23"
server_key_iv: "ca05feb99c296b35c6458160eb34c475"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.181974 9778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "67acb2bd14c0495c87afccd6c208a716"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "298455520c4566c71979cffb3df02a23"
server_key_iv: "ca05feb99c296b35c6458160eb34c475"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.183434 9778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:30.184315 9793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.184520 9778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:30.184616 9778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "67acb2bd14c0495c87afccd6c208a716"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "298455520c4566c71979cffb3df02a23"
server_key_iv: "ca05feb99c296b35c6458160eb34c475"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.184686 9778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:30.203274 9778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:30.205777 9778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:30.205958 9778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:30.206192 9778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:30.206514 9778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:30.206553 9778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.206584 9778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:30.206599 9778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.213124 9778 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:44963
I20260925 22:00:30.213189 9906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:44963 every 8 connection(s)
I20260925 22:00:30.213485 9778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:30.215804 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 9778
I20260925 22:00:30.215884 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:30.216019 9665 external_mini_cluster.cc:1428] Setting key 03ae7f78266f4ced3353e5d117da0009
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:30.223515 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.214246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:42463 (local address 127.9.112.126:40933)
0925 22:00:30.214308 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:30.214310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:30.214492 (+ 182us) server_negotiation.cc:407] Connection header received
0925 22:00:30.215045 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:30.215046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:30.215061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:30.215111 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:30.216100 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.216822 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.217823 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.218043 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.218099 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.220629 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:30.220630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:30.220631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:30.220638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:30.221990 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.222508 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.222509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.222509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.222556 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.222827 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.222827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.222827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.222963 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:30.223013 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:30.223373 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 22:00:30.223444 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:00:30.223780 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.214231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40933 (local address 127.9.112.65:42463)
0925 22:00:30.214369 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:30.214406 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:30.214934 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:30.215187 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:30.215191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:30.215767 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:30.216003 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:30.216009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.216911 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.216912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:30.217736 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:30.217738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.218116 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.218117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:30.218261 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.218886 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:30.218895 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:30.220518 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:30.222085 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:30.222094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:30.222097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:30.222430 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:30.222616 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:30.222617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:30.222617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:30.222763 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:30.223028 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:30.223035 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:30.223222 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 22:00:30.223354 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 22:00:30.223537 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
I20260925 22:00:30.224419 9907 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40933
I20260925 22:00:30.224520 9907 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:30.224709 9907 heartbeater.cc:511] Master 127.9.112.126:40933 requested a full tablet report, sending...
I20260925 22:00:30.225350 9717 ts_manager.cc:194] Registered new tserver with Master: 67acb2bd14c0495c87afccd6c208a716 (127.9.112.65:44963)
I20260925 22:00:30.226053 9717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:42463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:30.244019 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40933
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:30.329654 9912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:30.329916 9912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:30.329945 9912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:30.331149 9912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:30.331199 9912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:30.331228 9912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:30.332824 9912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:30.333050 9912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:30.333305 9912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:30.340494 9920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:30.340483 9918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:30.340480 9917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:30.341430 9912 server_base.cc:1034] running on GCE node
I20260925 22:00:30.341668 9912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:30.341950 9912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:30.343120 9912 hybrid_clock.cc:648] HybridClock initialized: now 1790373630343099 us; error 39 us; skew 500 ppm
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:30.345938 9912 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:30.346470 9912 webserver.cc:460] Webserver started at http://127.9.112.66:38815/ using document root <none> and password file <none>
I20260925 22:00:30.346695 9912 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:30.346745 9912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:30.346872 9912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:30.347914 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "55e23d43332a4e64a12bfb9cd63f3da6"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "9e791fbb23dab5cad55a7911a9a1dbb9"
server_key_iv: "978d8c1b68136c6204b8b3e5998438c6"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.348249 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "55e23d43332a4e64a12bfb9cd63f3da6"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "9e791fbb23dab5cad55a7911a9a1dbb9"
server_key_iv: "978d8c1b68136c6204b8b3e5998438c6"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.350044 9912 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260925 22:00:30.350934 9927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.351130 9912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:30.351246 9912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "55e23d43332a4e64a12bfb9cd63f3da6"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "9e791fbb23dab5cad55a7911a9a1dbb9"
server_key_iv: "978d8c1b68136c6204b8b3e5998438c6"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.351317 9912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:30.360555 9912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:30.363128 9912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:30.363286 9912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:30.363559 9912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:30.363895 9912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:30.363931 9912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.363958 9912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:30.363974 9912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.370008 9912 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:41385
I20260925 22:00:30.370148 10040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:41385 every 8 connection(s)
I20260925 22:00:30.370406 9912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:30.379292 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 9912
I20260925 22:00:30.379415 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:30.379601 9665 external_mini_cluster.cc:1428] Setting key b453359109f09fe0ff70533b838bf193
I20260925 22:00:30.380885 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.371128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:37575 (local address 127.9.112.126:40933)
0925 22:00:30.371206 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:30.371208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:30.371442 (+ 234us) server_negotiation.cc:407] Connection header received
0925 22:00:30.371940 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:30.371941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:30.371951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:30.372016 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:30.373036 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.373607 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.374771 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.374974 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.375037 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.377846 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:30.377847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:30.377848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:30.377856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:30.379202 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.379764 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.379766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.379767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.379838 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.380131 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.380131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.380132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.380247 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:30.380365 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:30.380702 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 22:00:30.380795 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:00:30.380991 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.371068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40933 (local address 127.9.112.66:37575)
0925 22:00:30.371283 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:30.371328 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:30.371831 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:30.372075 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:30.372079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:30.372725 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:30.372938 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:30.372943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.373716 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.373717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:30.374651 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:30.374654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.375072 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.375074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:30.375201 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.376117 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:30.376126 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:30.377744 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:30.379306 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:30.379315 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:30.379318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:30.379663 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:30.379913 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:30.379914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:30.379915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:30.380066 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:30.380364 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:30.380371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:30.380549 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 22:00:30.380653 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 22:00:30.380801 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":64,"threads_started":1}
I20260925 22:00:30.381635 10041 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40933
I20260925 22:00:30.381767 10041 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:30.381953 10041 heartbeater.cc:511] Master 127.9.112.126:40933 requested a full tablet report, sending...
I20260925 22:00:30.382344 9717 ts_manager.cc:194] Registered new tserver with Master: 55e23d43332a4e64a12bfb9cd63f3da6 (127.9.112.66:41385)
I20260925 22:00:30.382938 9717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:37575
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:30.408553 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40933
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:30.491540 10046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:30.491775 10046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:30.491819 10046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:30.493160 10046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:30.493211 10046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:30.493251 10046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:30.494812 10046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:30.495064 10046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:30.495288 10046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:30.500330 10054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:30.500334 10051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:30.500447 10052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:30.500483 10046 server_base.cc:1034] running on GCE node
I20260925 22:00:30.500833 10046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:30.501086 10046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:30.502234 10046 hybrid_clock.cc:648] HybridClock initialized: now 1790373630502218 us; error 36 us; skew 500 ppm
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:30.504851 10046 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:30.505339 10046 webserver.cc:460] Webserver started at http://127.9.112.67:39513/ using document root <none> and password file <none>
I20260925 22:00:30.505554 10046 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:30.505604 10046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:30.505730 10046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:30.506832 10046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "659f6d7022a8488fa27c390074d98793"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "bbf674966718a46d187fed55feb77a87"
server_key_iv: "d30ada371d42db21bfd8600f24eaee2c"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.507192 10046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "659f6d7022a8488fa27c390074d98793"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "bbf674966718a46d187fed55feb77a87"
server_key_iv: "d30ada371d42db21bfd8600f24eaee2c"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.508636 10046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:00:30.509438 10061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.509594 10046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:30.509677 10046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "659f6d7022a8488fa27c390074d98793"
format_stamp: "Formatted at 2026-09-25 22:00:30 on dist-test-slave-794d"
server_key: "bbf674966718a46d187fed55feb77a87"
server_key_iv: "d30ada371d42db21bfd8600f24eaee2c"
server_key_version: "encryptionkey@0"
I20260925 22:00:30.509739 10046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:30.522117 10046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:30.524248 10046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:30.524421 10046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:30.524639 10046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:30.524977 10046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:30.525017 10046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.525046 10046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:30.525071 10046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:30.531302 10046 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:43653
I20260925 22:00:30.531381 10174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:43653 every 8 connection(s)
I20260925 22:00:30.531723 10046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:30.533701 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10046
I20260925 22:00:30.533794 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:30.533954 9665 external_mini_cluster.cc:1428] Setting key 91dc5ebc4d328e473255c77fd49d50ad
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:30.543574 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.532496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:35683 (local address 127.9.112.126:40933)
0925 22:00:30.532541 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:30.532543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:30.532818 (+ 275us) server_negotiation.cc:407] Connection header received
0925 22:00:30.533461 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:30.533462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:30.533477 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:30.533528 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:30.534781 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.535486 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.537848 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.538137 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.538238 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.540995 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:30.540997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:30.540998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:30.541007 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:30.542225 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.542636 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.542637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.542637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.542685 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.542976 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.542976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.542977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.543083 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:30.543141 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:30.543432 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 22:00:30.543502 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:00:30.543887 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.532516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40933 (local address 127.9.112.67:35683)
0925 22:00:30.532666 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:30.532726 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:30.533346 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:30.533604 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:30.533608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:30.534368 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:30.534671 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:30.534680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.536102 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.536104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:30.537659 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:30.537663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.538256 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.538258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:30.538414 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.539030 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:30.539039 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:30.540849 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:30.542321 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:30.542328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:30.542331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:30.542572 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:30.542767 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:30.542768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:30.542768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:30.542909 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:30.543155 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:30.543161 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:30.543303 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 22:00:30.543398 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 22:00:30.543534 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260925 22:00:30.544397 10175 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40933
I20260925 22:00:30.544499 10175 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:30.544677 10175 heartbeater.cc:511] Master 127.9.112.126:40933 requested a full tablet report, sending...
I20260925 22:00:30.545084 9717 ts_manager.cc:194] Registered new tserver with Master: 659f6d7022a8488fa27c390074d98793 (127.9.112.67:43653)
I20260925 22:00:30.545501 9717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:35683
I20260925 22:00:30.546459 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:30.570143 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.561267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.9.112.65:44963)
0925 22:00:30.561424 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:30.561429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:30.561440 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:00:30.561466 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:30.561467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:30.561481 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:30.561544 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:30.562079 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.562856 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.563663 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.563997 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.564054 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.566576 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:30.566587 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:30.566587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:30.566595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:30.568875 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.569275 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.569278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.569279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.569342 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.569582 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.569582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.569583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.569742 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:30.569919 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:30.569980 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 22:00:30.570061 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
W20260925 22:00:30.570519 9821 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34848
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[9672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:30.593556 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:30.586276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.9.112.65:44963)
0925 22:00:30.586352 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:30.586354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:30.586366 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:00:30.586435 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:30.586436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:30.586445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:30.586510 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:30.587114 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.587654 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.588460 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:30.588672 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:30.588727 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:30.590947 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:30.590948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:30.590949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:30.590956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:30.592264 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.592662 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.592664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.592664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.592749 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:30.593010 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:30.593011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:30.593012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:30.593128 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:30.593245 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:30.593404 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 22:00:30.593486 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 22:00:30.594177 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 9778
I20260925 22:00:30.598786 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 9912
I20260925 22:00:30.602386 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10046
I20260925 22:00:30.605902 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 9685
2026-09-25T22:00:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1842 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[10189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[10189](info): set up 2 sockets
Sep 25 22:00:30 dist-test-slave-794d krb5kdc[10189](info): commencing operation
krb5kdc: starting...
W20260925 22:00:31.563791 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:31 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:31.684196 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35727
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:31.766760 10202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:31.766992 10202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:31.767014 10202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:31.768247 10202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:31.768296 10202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:31.768309 10202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:31.768321 10202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:31.768332 10202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:31.769904 10202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:31.770114 10202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:31.770298 10202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:00:31.775255 10202 server_base.cc:1034] running on GCE node
W20260925 22:00:31.775220 10208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:31.775207 10210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:31.775233 10207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:31.775642 10202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:31.775943 10202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:31.777094 10202 hybrid_clock.cc:648] HybridClock initialized: now 1790373631777079 us; error 32 us; skew 500 ppm
Sep 25 22:00:31 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373631, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:31.779632 10202 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:31.780052 10202 webserver.cc:460] Webserver started at http://127.9.112.126:44321/ using document root <none> and password file <none>
I20260925 22:00:31.780262 10202 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:31.780332 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:31.780480 10202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:31.781555 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "235e10570ebc41eb9a1e4c61efbb892b"
format_stamp: "Formatted at 2026-09-25 22:00:31 on dist-test-slave-794d"
server_key: "70587b30900516777b8585e2f9e670c2"
server_key_iv: "fc8fd4a62b0284c369032da1586a0932"
server_key_version: "encryptionkey@0"
I20260925 22:00:31.781909 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "235e10570ebc41eb9a1e4c61efbb892b"
format_stamp: "Formatted at 2026-09-25 22:00:31 on dist-test-slave-794d"
server_key: "70587b30900516777b8585e2f9e670c2"
server_key_iv: "fc8fd4a62b0284c369032da1586a0932"
server_key_version: "encryptionkey@0"
I20260925 22:00:31.783208 10202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:31.783948 10217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:31.784117 10202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:31.784204 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "235e10570ebc41eb9a1e4c61efbb892b"
format_stamp: "Formatted at 2026-09-25 22:00:31 on dist-test-slave-794d"
server_key: "70587b30900516777b8585e2f9e670c2"
server_key_iv: "fc8fd4a62b0284c369032da1586a0932"
server_key_version: "encryptionkey@0"
I20260925 22:00:31.784265 10202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:31.794119 10202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:31.796069 10202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:31.796206 10202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:31.800030 10202 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35727
I20260925 22:00:31.800081 10269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35727 every 8 connection(s)
I20260925 22:00:31.800470 10202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:31.801362 10270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:31.803776 10270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b: Bootstrap starting.
I20260925 22:00:31.804414 10270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:31.804682 10270 log.cc:824] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:31.805409 10270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b: No bootstrap required, opened a new log
I20260925 22:00:31.807003 10270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235e10570ebc41eb9a1e4c61efbb892b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35727 } }
I20260925 22:00:31.807159 10270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:31.807181 10270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235e10570ebc41eb9a1e4c61efbb892b, State: Initialized, Role: FOLLOWER
I20260925 22:00:31.807332 10270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235e10570ebc41eb9a1e4c61efbb892b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35727 } }
I20260925 22:00:31.807405 10270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:31.807430 10270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:31.807477 10270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:31.808027 10270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235e10570ebc41eb9a1e4c61efbb892b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35727 } }
I20260925 22:00:31.808126 10270 leader_election.cc:302] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 235e10570ebc41eb9a1e4c61efbb892b; no voters:
I20260925 22:00:31.808305 10270 leader_election.cc:288] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:31.808411 10273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:31.808655 10270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:31.808586 10273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [term 1 LEADER]: Becoming Leader. State: Replica: 235e10570ebc41eb9a1e4c61efbb892b, State: Running, Role: LEADER
I20260925 22:00:31.808773 10273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235e10570ebc41eb9a1e4c61efbb892b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35727 } }
I20260925 22:00:31.808800 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10202
I20260925 22:00:31.808879 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:31.809051 9665 external_mini_cluster.cc:1428] Setting key 5a72511aba2f3c5d51afafc8d3cc5ae8
I20260925 22:00:31.809176 10275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 235e10570ebc41eb9a1e4c61efbb892b. Latest consensus state: current_term: 1 leader_uuid: "235e10570ebc41eb9a1e4c61efbb892b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235e10570ebc41eb9a1e4c61efbb892b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35727 } } }
I20260925 22:00:31.809262 10275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:31.809406 10274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "235e10570ebc41eb9a1e4c61efbb892b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235e10570ebc41eb9a1e4c61efbb892b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35727 } } }
I20260925 22:00:31.809473 10274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:31.809602 10277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:31.810148 10277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:31.813014 10277 catalog_manager.cc:1347] Generated new cluster ID: 36f0c3b2b5ea4246b93ea50d1101fd7c
I20260925 22:00:31.813076 10277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:31 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:31.819527 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:31.809768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.9.112.126:35727)
0925 22:00:31.810150 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:31.810161 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:31.810196 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:00:31.810453 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:31.810460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:31.810520 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:31.810668 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:31.811452 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:31.812848 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:31.813608 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:31.814046 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:31.814145 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:31.816272 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:31.816276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:31.816276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:31.816289 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:31.818025 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:31.818409 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:31.818411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:31.818412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:31.818515 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:31.818694 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:31.818695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:31.818695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:31.818969 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:31.819121 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:31.819221 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 22:00:31.819362 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":69,"threads_started":1}
I20260925 22:00:31.821758 10277 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:31.822288 10277 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:31.833086 10277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 235e10570ebc41eb9a1e4c61efbb892b: Generated new TSK 0
I20260925 22:00:31.833328 10277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:31.871821 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35727
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:31.961054 10296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:31.961299 10296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:31.961335 10296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:31.962634 10296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:31.962699 10296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:31.962738 10296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:31.964447 10296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:31.964717 10296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:31.964955 10296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:31.970005 10302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:31.970031 10304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:31.970111 10301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:31.970330 10296 server_base.cc:1034] running on GCE node
I20260925 22:00:31.970552 10296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:31.970765 10296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:31.971916 10296 hybrid_clock.cc:648] HybridClock initialized: now 1790373631971902 us; error 34 us; skew 500 ppm
Sep 25 22:00:31 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373631, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:31.974526 10296 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:31.974998 10296 webserver.cc:460] Webserver started at http://127.9.112.65:40505/ using document root <none> and password file <none>
I20260925 22:00:31.975208 10296 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:31.975253 10296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:31.975378 10296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:31.976526 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "75f20295ded14bb8a7a771ec5240eaee"
format_stamp: "Formatted at 2026-09-25 22:00:31 on dist-test-slave-794d"
server_key: "4069d3151b8b725804898690ee24a70e"
server_key_iv: "7f47f2df275abff7b35bb7acb275a6f4"
server_key_version: "encryptionkey@0"
I20260925 22:00:31.976856 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "75f20295ded14bb8a7a771ec5240eaee"
format_stamp: "Formatted at 2026-09-25 22:00:31 on dist-test-slave-794d"
server_key: "4069d3151b8b725804898690ee24a70e"
server_key_iv: "7f47f2df275abff7b35bb7acb275a6f4"
server_key_version: "encryptionkey@0"
I20260925 22:00:31.978204 10296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:31.979235 10311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:31.979450 10296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:31.979544 10296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "75f20295ded14bb8a7a771ec5240eaee"
format_stamp: "Formatted at 2026-09-25 22:00:31 on dist-test-slave-794d"
server_key: "4069d3151b8b725804898690ee24a70e"
server_key_iv: "7f47f2df275abff7b35bb7acb275a6f4"
server_key_version: "encryptionkey@0"
I20260925 22:00:31.979626 10296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:31.988435 10296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:31.990628 10296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:31.990763 10296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:31.991006 10296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:31.991309 10296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:31.991341 10296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:31.991364 10296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:31.991387 10296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:31.997076 10296 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:44621
I20260925 22:00:31.997138 10424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:44621 every 8 connection(s)
I20260925 22:00:31.997428 10296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:00:31 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373631, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:32.006985 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10296
I20260925 22:00:32.007073 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:32.007241 9665 external_mini_cluster.cc:1428] Setting key 6a43f93f31a158722ea3acbac40e8d24
I20260925 22:00:32.007297 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:31.998124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:37295 (local address 127.9.112.126:35727)
0925 22:00:31.998206 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:31.998209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:31.998348 (+ 139us) server_negotiation.cc:407] Connection header received
0925 22:00:31.999316 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:31.999317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:31.999336 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:31.999396 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.000101 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.000709 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.001870 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.002111 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.002178 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.004550 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.004552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.004552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.004560 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.005934 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.006374 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.006375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.006376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.006431 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.006635 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.006635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.006635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.006742 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.006789 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.007096 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.007203 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:00:32.007514 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:31.998096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35727 (local address 127.9.112.65:37295)
0925 22:00:31.998229 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:31.998265 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:31.999190 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:31.999470 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:31.999473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:31.999736 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:31.999981 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:31.999986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.000823 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.000824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.001723 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.001726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.002219 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.002220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.002355 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.002927 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:32.002937 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:32.004449 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:32.006023 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:32.006030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:32.006033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:32.006299 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:32.006485 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:32.006486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:32.006486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:32.006591 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:32.006814 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:32.006820 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:32.006959 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.007051 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.007253 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260925 22:00:32.008070 10425 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35727
I20260925 22:00:32.008175 10425 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:32.008401 10425 heartbeater.cc:511] Master 127.9.112.126:35727 requested a full tablet report, sending...
I20260925 22:00:32.008927 10234 ts_manager.cc:194] Registered new tserver with Master: 75f20295ded14bb8a7a771ec5240eaee (127.9.112.65:44621)
I20260925 22:00:32.009544 10234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:37295
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:32.035652 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35727
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:32.118306 10430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:32.118542 10430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:32.118569 10430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:32.119818 10430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:32.119867 10430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:32.119895 10430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:32.121567 10430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:32.121834 10430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:32.122064 10430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:32.127364 10438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:32.127408 10435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:32.127364 10436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:32.127578 10430 server_base.cc:1034] running on GCE node
I20260925 22:00:32.127898 10430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:32.128153 10430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:32.129391 10430 hybrid_clock.cc:648] HybridClock initialized: now 1790373632129370 us; error 41 us; skew 500 ppm
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:32.132685 10430 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:32.133279 10430 webserver.cc:460] Webserver started at http://127.9.112.66:41039/ using document root <none> and password file <none>
I20260925 22:00:32.133507 10430 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:32.133558 10430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:32.133692 10430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:32.134788 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "0ad9406b3d4640b9ac05658ac6d91e31"
format_stamp: "Formatted at 2026-09-25 22:00:32 on dist-test-slave-794d"
server_key: "019b38fe86ff3499564571165c15956a"
server_key_iv: "38dc9ed27d1db0ab7d419191fd2a51ae"
server_key_version: "encryptionkey@0"
I20260925 22:00:32.135162 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "0ad9406b3d4640b9ac05658ac6d91e31"
format_stamp: "Formatted at 2026-09-25 22:00:32 on dist-test-slave-794d"
server_key: "019b38fe86ff3499564571165c15956a"
server_key_iv: "38dc9ed27d1db0ab7d419191fd2a51ae"
server_key_version: "encryptionkey@0"
I20260925 22:00:32.136894 10430 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260925 22:00:32.138000 10445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:32.138300 10430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:32.138419 10430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "0ad9406b3d4640b9ac05658ac6d91e31"
format_stamp: "Formatted at 2026-09-25 22:00:32 on dist-test-slave-794d"
server_key: "019b38fe86ff3499564571165c15956a"
server_key_iv: "38dc9ed27d1db0ab7d419191fd2a51ae"
server_key_version: "encryptionkey@0"
I20260925 22:00:32.138502 10430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:32.182423 10430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:32.184691 10430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:32.184830 10430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:32.185045 10430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:32.185331 10430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:32.185364 10430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:32.185403 10430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:32.185426 10430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:32.191061 10430 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:46665
I20260925 22:00:32.191109 10558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:46665 every 8 connection(s)
I20260925 22:00:32.191434 10430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:32.200555 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.192175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:45011 (local address 127.9.112.126:35727)
0925 22:00:32.192260 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.192262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.192437 (+ 175us) server_negotiation.cc:407] Connection header received
0925 22:00:32.193432 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.193434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.193447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.193500 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.194124 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.194634 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.195605 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.195813 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.195870 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.198018 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.198020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.198020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.198029 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.199177 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.199596 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.199597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.199597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.199656 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.199882 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.199882 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.199883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.200004 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.200062 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.200333 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.200458 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 22:00:32.200734 10561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.192096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35727 (local address 127.9.112.66:45011)
0925 22:00:32.192284 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.192331 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.193312 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.193573 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.193576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.193838 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:32.194036 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.194041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.194715 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.194716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.195524 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.195526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.195883 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.195884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.196003 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.196546 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:32.196555 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:32.197918 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:32.199259 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:32.199267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:32.199269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:32.199524 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:32.199712 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:32.199712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:32.199713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:32.199818 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:32.200062 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:32.200068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:32.200212 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.200306 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.200500 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260925 22:00:32.201215 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10430
I20260925 22:00:32.201303 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:32.201282 10559 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35727
I20260925 22:00:32.201395 10559 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:32.201510 9665 external_mini_cluster.cc:1428] Setting key 2bb112d4acd51eb37c6f5b3c763fbf40
I20260925 22:00:32.201588 10559 heartbeater.cc:511] Master 127.9.112.126:35727 requested a full tablet report, sending...
I20260925 22:00:32.202024 10233 ts_manager.cc:194] Registered new tserver with Master: 0ad9406b3d4640b9ac05658ac6d91e31 (127.9.112.66:46665)
I20260925 22:00:32.202488 10233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:45011
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:32.228224 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35727
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:32.308575 10564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:32.308806 10564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:32.308832 10564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:32.310199 10564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:32.310264 10564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:32.310307 10564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:32.312021 10564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:32.312266 10564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:32.312533 10564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:32.317890 10570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:32.317947 10569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:32.317963 10564 server_base.cc:1034] running on GCE node
W20260925 22:00:32.317911 10572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:32.318269 10564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:32.318514 10564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:32.319671 10564 hybrid_clock.cc:648] HybridClock initialized: now 1790373632319652 us; error 27 us; skew 500 ppm
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:32.322394 10564 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:32.322852 10564 webserver.cc:460] Webserver started at http://127.9.112.67:41625/ using document root <none> and password file <none>
I20260925 22:00:32.323055 10564 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:32.323100 10564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:32.323210 10564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:32.324445 10564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "2ec723fe92e64f0ea739fd6dec1758ee"
format_stamp: "Formatted at 2026-09-25 22:00:32 on dist-test-slave-794d"
server_key: "ead29a66b531c79ccfb56f127a5d40a1"
server_key_iv: "24950eb70a5d12912960a4e287c3c0cf"
server_key_version: "encryptionkey@0"
I20260925 22:00:32.324790 10564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "2ec723fe92e64f0ea739fd6dec1758ee"
format_stamp: "Formatted at 2026-09-25 22:00:32 on dist-test-slave-794d"
server_key: "ead29a66b531c79ccfb56f127a5d40a1"
server_key_iv: "24950eb70a5d12912960a4e287c3c0cf"
server_key_version: "encryptionkey@0"
I20260925 22:00:32.326072 10564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:32.326840 10579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:32.327005 10564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:32.327083 10564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "2ec723fe92e64f0ea739fd6dec1758ee"
format_stamp: "Formatted at 2026-09-25 22:00:32 on dist-test-slave-794d"
server_key: "ead29a66b531c79ccfb56f127a5d40a1"
server_key_iv: "24950eb70a5d12912960a4e287c3c0cf"
server_key_version: "encryptionkey@0"
I20260925 22:00:32.327153 10564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:32.351862 10564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:32.354175 10564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:32.354303 10564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:32.354543 10564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:32.354853 10564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:32.354893 10564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:32.354916 10564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:32.354938 10564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:32.360716 10564 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:35767
I20260925 22:00:32.360770 10692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:35767 every 8 connection(s)
I20260925 22:00:32.361073 10564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:32.363019 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10564
I20260925 22:00:32.363121 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:32.363270 9665 external_mini_cluster.cc:1428] Setting key c0f8b04c9f1bedb6e59f453850776a8b
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:32.370715 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.361783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:49611 (local address 127.9.112.126:35727)
0925 22:00:32.361868 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.361870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.362045 (+ 175us) server_negotiation.cc:407] Connection header received
0925 22:00:32.363053 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.363055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.363068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.363146 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.363995 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.364584 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.365577 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.365770 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.365825 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.367891 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.367892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.367892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.367898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.369227 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.369652 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.369653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.369653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.369711 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.370021 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.370022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.370022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.370146 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.370209 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.370559 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.370645 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 22:00:32.370860 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.361759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35727 (local address 127.9.112.67:49611)
0925 22:00:32.361913 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.361951 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.362924 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.363271 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.363275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.363634 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:32.363920 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.363927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.364677 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.364678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.365502 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.365504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.365835 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.365836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.365938 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.366402 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:32.366410 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:32.367810 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:32.369342 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:32.369349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:32.369352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:32.369582 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:32.369790 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:32.369790 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:32.369791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:32.369903 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:32.370230 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:32.370236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:32.370401 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.370505 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.370645 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260925 22:00:32.371451 10693 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35727
I20260925 22:00:32.371552 10693 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:32.371704 10693 heartbeater.cc:511] Master 127.9.112.126:35727 requested a full tablet report, sending...
I20260925 22:00:32.372083 10233 ts_manager.cc:194] Registered new tserver with Master: 2ec723fe92e64f0ea739fd6dec1758ee (127.9.112.67:35767)
I20260925 22:00:32.372510 10233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:49611
I20260925 22:00:32.374977 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:32.383348 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.376514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.9.112.126:35727)
0925 22:00:32.376568 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.376570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.376711 (+ 141us) server_negotiation.cc:407] Connection header received
0925 22:00:32.376787 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.376788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.376797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.376844 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.377368 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.377869 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.378564 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.378776 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.378831 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.380101 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.380103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.380103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.380112 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.381696 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.382253 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.382254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.382254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.382319 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.382504 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.382505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.382505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.382674 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.382765 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.383077 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.383242 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:00:32.385882 10233 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:32.387547 10233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:32.394469 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.390521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.9.112.65:44621)
0925 22:00:32.390702 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.390708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.390766 (+ 58us) server_negotiation.cc:407] Connection header received
0925 22:00:32.390848 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.390849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.390862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.390929 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:32.391127 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.391951 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.393374 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.394067 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.394132 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.394193 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.394337 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.394378 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260925 22:00:32.394457 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.390218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44621 (local address 127.0.0.1:59558)
0925 22:00:32.390700 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.390711 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.390778 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.390953 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.390957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.390959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:32.391040 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.391042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.392047 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.392048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.393289 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.393292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.394139 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.394140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.394229 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.394243 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.394312 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.394371 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260925 22:00:32.394994 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.390546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.9.112.66:46665)
0925 22:00:32.390707 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.390710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.391148 (+ 438us) server_negotiation.cc:407] Connection header received
0925 22:00:32.391189 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.391191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.391206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.391267 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:32.391454 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.392512 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.393665 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.394374 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.394428 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.394483 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.394878 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.394922 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260925 22:00:32.395048 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.390218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:35767 (local address 127.0.0.1:35404)
0925 22:00:32.390392 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.390436 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.390500 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.391073 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.391074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.391077 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:32.391130 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.391132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.392414 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.392415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.393487 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.393489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.394446 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.394447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.394509 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.394870 (+ 361us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.394911 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.394945 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":24,"thread_start_us":72,"threads_started":1}
I20260925 22:00:32.394985 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.390427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:46665 (local address 127.0.0.1:52052)
0925 22:00:32.391046 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.391055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.391099 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.391342 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.391343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.391345 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:32.391377 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.391379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.392591 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.392592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.393566 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.393569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.394511 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.394511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.394791 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.394796 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.394853 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.394895 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":67,"threads_started":1}
I20260925 22:00:32.395259 10359 tablet_service.cc:1467] Processing CreateTablet for tablet 2a56072b81984d22b3a8eb280749087d (DEFAULT_TABLE table=test-table [id=ed2a6a9ac88b44cd93a76deedb8099ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:32.395537 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.390338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.9.112.67:35767)
0925 22:00:32.390800 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.390805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.390814 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:32.390854 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.390856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.390870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.390943 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:32.391211 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.392300 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.393564 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.394283 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.394353 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.394411 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.395028 (+ 617us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.395424 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":82,"threads_started":1}
I20260925 22:00:32.395665 10359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a56072b81984d22b3a8eb280749087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:32.395805 10493 tablet_service.cc:1467] Processing CreateTablet for tablet 2a56072b81984d22b3a8eb280749087d (DEFAULT_TABLE table=test-table [id=ed2a6a9ac88b44cd93a76deedb8099ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:32.396085 10493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a56072b81984d22b3a8eb280749087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:32.396975 10627 tablet_service.cc:1467] Processing CreateTablet for tablet 2a56072b81984d22b3a8eb280749087d (DEFAULT_TABLE table=test-table [id=ed2a6a9ac88b44cd93a76deedb8099ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:32.397313 10627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a56072b81984d22b3a8eb280749087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:32.398489 10710 tablet_bootstrap.cc:492] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee: Bootstrap starting.
I20260925 22:00:32.399027 10710 tablet_bootstrap.cc:654] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:32.399516 10711 tablet_bootstrap.cc:492] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: Bootstrap starting.
I20260925 22:00:32.400029 10710 log.cc:824] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:32.400167 10711 tablet_bootstrap.cc:654] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:32.400358 10712 tablet_bootstrap.cc:492] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee: Bootstrap starting.
I20260925 22:00:32.400660 10711 log.cc:824] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:32.400905 10712 tablet_bootstrap.cc:654] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:32.401296 10711 tablet_bootstrap.cc:492] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: No bootstrap required, opened a new log
I20260925 22:00:32.401351 10711 ts_tablet_manager.cc:1397] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:32.401346 10712 log.cc:824] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:32.402048 10710 tablet_bootstrap.cc:492] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee: No bootstrap required, opened a new log
I20260925 22:00:32.402065 10712 tablet_bootstrap.cc:492] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee: No bootstrap required, opened a new log
I20260925 22:00:32.402123 10712 ts_tablet_manager.cc:1397] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:32.402122 10710 ts_tablet_manager.cc:1397] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 22:00:32.403277 10711 raft_consensus.cc:348] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.403486 10711 raft_consensus.cc:374] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:32.403520 10711 raft_consensus.cc:729] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad9406b3d4640b9ac05658ac6d91e31, State: Initialized, Role: FOLLOWER
I20260925 22:00:32.403649 10711 consensus_queue.cc:260] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.403769 10710 raft_consensus.cc:348] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.403954 10710 raft_consensus.cc:374] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:32.403951 10711 ts_tablet_manager.cc:1428] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:32.403959 10559 heartbeater.cc:503] Master 127.9.112.126:35727 was elected leader, sending a full tablet report...
I20260925 22:00:32.403999 10710 raft_consensus.cc:729] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f20295ded14bb8a7a771ec5240eaee, State: Initialized, Role: FOLLOWER
I20260925 22:00:32.404115 10710 consensus_queue.cc:260] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.404409 10710 ts_tablet_manager.cc:1428] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:00:32.404260 10712 raft_consensus.cc:348] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.404439 10425 heartbeater.cc:503] Master 127.9.112.126:35727 was elected leader, sending a full tablet report...
I20260925 22:00:32.404474 10712 raft_consensus.cc:374] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:32.404507 10712 raft_consensus.cc:729] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec723fe92e64f0ea739fd6dec1758ee, State: Initialized, Role: FOLLOWER
I20260925 22:00:32.404613 10712 consensus_queue.cc:260] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.404865 10712 ts_tablet_manager.cc:1428] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:32.404991 10693 heartbeater.cc:503] Master 127.9.112.126:35727 was elected leader, sending a full tablet report...
I20260925 22:00:32.417915 10716 raft_consensus.cc:482] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:32.418051 10716 raft_consensus.cc:504] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.418443 10716 leader_election.cc:288] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ec723fe92e64f0ea739fd6dec1758ee (127.9.112.67:35767), 75f20295ded14bb8a7a771ec5240eaee (127.9.112.65:44621)
I20260925 22:00:32.421540 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.418709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:52865 (local address 127.9.112.65:44621)
0925 22:00:32.418754 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.418756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.418900 (+ 144us) server_negotiation.cc:407] Connection header received
0925 22:00:32.418940 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.418941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.418954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.419003 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:32.419124 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.419592 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.420719 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.421118 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.421175 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.421184 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.421443 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.421485 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:00:32.421573 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.418633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44621 (local address 127.9.112.66:52865)
0925 22:00:32.418827 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.418836 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.418895 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.419005 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.419005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.419007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:32.419064 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.419065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.419668 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.419669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.420623 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.420626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.421212 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.421213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.421297 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.421303 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.421418 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.421472 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":35,"thread_start_us":120,"threads_started":1}
I20260925 22:00:32.421981 10561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.418615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:35767 (local address 127.9.112.66:40123)
0925 22:00:32.418681 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:32.418694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:32.418740 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:32.418953 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:32.418953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:32.418955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:32.419038 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.419041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.419668 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.419669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.420625 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:32.420628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.421758 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.421759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:32.421824 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.421827 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:32.421879 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:00:32.421918 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":73}
I20260925 22:00:32.421998 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.418698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:40123 (local address 127.9.112.67:35767)
0925 22:00:32.418757 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.418758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.418768 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:32.418874 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.418875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.418884 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.418939 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:32.419097 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.419588 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.420714 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.421653 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.421720 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.421729 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.421899 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.421941 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:00:32.422091 10379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a56072b81984d22b3a8eb280749087d" candidate_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f20295ded14bb8a7a771ec5240eaee" is_pre_election: true
I20260925 22:00:32.422272 10379 raft_consensus.cc:2454] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad9406b3d4640b9ac05658ac6d91e31 in term 0.
I20260925 22:00:32.422269 10647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a56072b81984d22b3a8eb280749087d" candidate_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" is_pre_election: true
I20260925 22:00:32.422410 10647 raft_consensus.cc:2454] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad9406b3d4640b9ac05658ac6d91e31 in term 0.
I20260925 22:00:32.422511 10447 leader_election.cc:302] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad9406b3d4640b9ac05658ac6d91e31, 75f20295ded14bb8a7a771ec5240eaee; no voters:
I20260925 22:00:32.422638 10716 raft_consensus.cc:2781] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:32.422700 10716 raft_consensus.cc:482] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:32.422716 10716 raft_consensus.cc:3037] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:32.423398 10716 raft_consensus.cc:504] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.423522 10716 leader_election.cc:288] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [CANDIDATE]: Term 1 election: Requested vote from peers 2ec723fe92e64f0ea739fd6dec1758ee (127.9.112.67:35767), 75f20295ded14bb8a7a771ec5240eaee (127.9.112.65:44621)
I20260925 22:00:32.423710 10379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a56072b81984d22b3a8eb280749087d" candidate_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f20295ded14bb8a7a771ec5240eaee"
I20260925 22:00:32.423702 10647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a56072b81984d22b3a8eb280749087d" candidate_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec723fe92e64f0ea739fd6dec1758ee"
I20260925 22:00:32.423789 10379 raft_consensus.cc:3037] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:32.423852 10647 raft_consensus.cc:3037] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:32.424523 10379 raft_consensus.cc:2454] T 2a56072b81984d22b3a8eb280749087d P 75f20295ded14bb8a7a771ec5240eaee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad9406b3d4640b9ac05658ac6d91e31 in term 1.
I20260925 22:00:32.424675 10447 leader_election.cc:302] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad9406b3d4640b9ac05658ac6d91e31, 75f20295ded14bb8a7a771ec5240eaee; no voters:
I20260925 22:00:32.424687 10647 raft_consensus.cc:2454] T 2a56072b81984d22b3a8eb280749087d P 2ec723fe92e64f0ea739fd6dec1758ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad9406b3d4640b9ac05658ac6d91e31 in term 1.
I20260925 22:00:32.424782 10716 raft_consensus.cc:2781] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:32.424948 10716 raft_consensus.cc:686] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad9406b3d4640b9ac05658ac6d91e31, State: Running, Role: LEADER
I20260925 22:00:32.425045 10716 consensus_queue.cc:237] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } }
I20260925 22:00:32.425999 10233 catalog_manager.cc:5495] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad9406b3d4640b9ac05658ac6d91e31 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ec723fe92e64f0ea739fd6dec1758ee" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f20295ded14bb8a7a771ec5240eaee" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad9406b3d4640b9ac05658ac6d91e31" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 46665 } health_report { overall_health: HEALTHY } } }
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
W20260925 22:00:32.441980 10560 tablet.cc:2369] T 2a56072b81984d22b3a8eb280749087d P 0ad9406b3d4640b9ac05658ac6d91e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:32.444561 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.436897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59560 (local address 127.9.112.65:44621)
0925 22:00:32.436973 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.436976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.437092 (+ 116us) server_negotiation.cc:407] Connection header received
0925 22:00:32.437194 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.437195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.437206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.437258 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.437849 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.438331 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.439119 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.439318 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.439382 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.441727 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.441744 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.441744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.441753 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.443375 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.443694 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.443697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.443698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.443755 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.443977 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.443978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.443978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.444139 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.444293 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.444374 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.444467 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:32.467365 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.459845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.9.112.65:44621)
0925 22:00:32.459928 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.459930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.459976 (+ 46us) server_negotiation.cc:407] Connection header received
0925 22:00:32.460038 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.460039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.460049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.460100 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.460700 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.461182 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.461987 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.462214 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.462291 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.464646 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.464648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.464649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.464658 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.466107 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.466459 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.466460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.466460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.466520 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.466729 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.466730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.466730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.466837 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.466908 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.467160 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.467233 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:32.490799 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:32.483169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.9.112.65:44621)
0925 22:00:32.483266 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:32.483268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:32.483407 (+ 139us) server_negotiation.cc:407] Connection header received
0925 22:00:32.483439 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:32.483441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:32.483453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:32.483493 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:32.484154 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.484812 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.485600 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:32.485795 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:32.485866 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:32.488240 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:32.488242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:32.488242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:32.488250 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:32.489718 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.490139 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.490139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.490140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.490194 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:32.490357 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:32.490358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:32.490358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:32.490461 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:32.490533 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:32.490635 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 22:00:32.490693 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 22:00:32.491287 10339 tablet_service.cc:2908] Rejecting scan request for tablet 2a56072b81984d22b3a8eb280749087d: Uninitialized: clean time has not yet been initialized
I20260925 22:00:32.492415 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10296
I20260925 22:00:32.496893 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10430
I20260925 22:00:32.500828 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10564
I20260925 22:00:32.504662 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10202
2026-09-25T22:00:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1899 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10734](info): set up 2 sockets
Sep 25 22:00:32 dist-test-slave-794d krb5kdc[10734](info): commencing operation
krb5kdc: starting...
W20260925 22:00:33.475920 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:33 dist-test-slave-794d krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:33.600152 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40981
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:33.681440 10747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:33.681679 10747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:33.681710 10747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:33.683004 10747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:33.683056 10747 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 22:00:33.683074 10747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:33.683089 10747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:33.683104 10747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:33.683120 10747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:33.684796 10747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40981
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:33.685066 10747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:33.685281 10747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:33.690373 10752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:33.690390 10747 server_base.cc:1034] running on GCE node
W20260925 22:00:33.690405 10755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:33.690412 10753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:33.690855 10747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:33.691157 10747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:33.692323 10747 hybrid_clock.cc:648] HybridClock initialized: now 1790373633692300 us; error 35 us; skew 500 ppm
Sep 25 22:00:33 dist-test-slave-794d krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373633, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:33.695014 10747 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:33.695515 10747 webserver.cc:460] Webserver started at http://127.9.112.126:43891/ using document root <none> and password file <none>
I20260925 22:00:33.695726 10747 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:33.695776 10747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:33.695905 10747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:33.697010 10747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "0bbb0a52ca24476aa0fdbcd4cd310530"
format_stamp: "Formatted at 2026-09-25 22:00:33 on dist-test-slave-794d"
server_key: "b4d6bea5e9b88ea6002da23228326dd6"
server_key_iv: "98c2dedb9904ad5ecaba7767a17df792"
server_key_version: "encryptionkey@0"
I20260925 22:00:33.697345 10747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "0bbb0a52ca24476aa0fdbcd4cd310530"
format_stamp: "Formatted at 2026-09-25 22:00:33 on dist-test-slave-794d"
server_key: "b4d6bea5e9b88ea6002da23228326dd6"
server_key_iv: "98c2dedb9904ad5ecaba7767a17df792"
server_key_version: "encryptionkey@0"
I20260925 22:00:33.698598 10747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:33.699251 10762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:33.699393 10747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:33.699477 10747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "0bbb0a52ca24476aa0fdbcd4cd310530"
format_stamp: "Formatted at 2026-09-25 22:00:33 on dist-test-slave-794d"
server_key: "b4d6bea5e9b88ea6002da23228326dd6"
server_key_iv: "98c2dedb9904ad5ecaba7767a17df792"
server_key_version: "encryptionkey@0"
I20260925 22:00:33.699539 10747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:33.711771 10747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:33.713914 10747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:33.714063 10747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:33.718470 10747 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40981
I20260925 22:00:33.718602 10824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40981 every 8 connection(s)
I20260925 22:00:33.718891 10747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:33.719563 10825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:33.721983 10825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: Bootstrap starting.
I20260925 22:00:33.722545 10825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:33.722808 10825 log.cc:824] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:33.723515 10825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: No bootstrap required, opened a new log
I20260925 22:00:33.724824 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10747
I20260925 22:00:33.724910 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:33.725061 9665 external_mini_cluster.cc:1428] Setting key 9efc948fc392a48c2a078818021847fc
I20260925 22:00:33.725977 10825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40981 } }
I20260925 22:00:33.726150 10825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:33.726189 10825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbb0a52ca24476aa0fdbcd4cd310530, State: Initialized, Role: FOLLOWER
I20260925 22:00:33.726348 10825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40981 } }
I20260925 22:00:33.726418 10825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:33.726439 10825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:33.726485 10825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:33.727142 10825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40981 } }
I20260925 22:00:33.727252 10825 leader_election.cc:302] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bbb0a52ca24476aa0fdbcd4cd310530; no voters:
I20260925 22:00:33.727402 10825 leader_election.cc:288] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:33.727442 10830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:33.727583 10830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [term 1 LEADER]: Becoming Leader. State: Replica: 0bbb0a52ca24476aa0fdbcd4cd310530, State: Running, Role: LEADER
I20260925 22:00:33.727639 10825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:33.727721 10830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40981 } }
I20260925 22:00:33.728083 10830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bbb0a52ca24476aa0fdbcd4cd310530. Latest consensus state: current_term: 1 leader_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40981 } } }
I20260925 22:00:33.728304 10830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:33.728235 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbb0a52ca24476aa0fdbcd4cd310530" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40981 } } }
I20260925 22:00:33.728561 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:33.728727 10839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:33.729259 10839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:33.730824 10839 catalog_manager.cc:1347] Generated new cluster ID: 0cdbe5aafd4c4f6e95666b9c552e87b9
I20260925 22:00:33.730867 10839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:33 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:33.736159 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:33.725591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.9.112.126:40981)
0925 22:00:33.725802 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:33.725813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:33.725850 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:00:33.725973 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:33.725980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:33.726038 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:33.726200 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:33.726824 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.728203 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.729298 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.729744 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.729840 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:33.732540 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:33.732544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:33.732545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:33.732558 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:33.734502 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:33.734896 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:33.734900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:33.734900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:33.735033 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:33.735215 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:33.735215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:33.735216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:33.735539 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:33.735700 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:33.735772 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:00:33.735944 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260925 22:00:33.755086 10839 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:33.755633 10839 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:33.766060 10839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: Generated new TSK 0
I20260925 22:00:33.766243 10839 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:33.799716 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40981
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:33.882170 10851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:33.882402 10851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:33.882428 10851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:33.883697 10851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:33.883760 10851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:33.883790 10851 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:33.883814 10851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:33.885352 10851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:33.885581 10851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:33.885812 10851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:33.891234 10857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:33.891247 10851 server_base.cc:1034] running on GCE node
W20260925 22:00:33.891229 10856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:33.891222 10859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:33.891745 10851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:33.891997 10851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:33.893141 10851 hybrid_clock.cc:648] HybridClock initialized: now 1790373633893118 us; error 41 us; skew 500 ppm
Sep 25 22:00:33 dist-test-slave-794d krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373633, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:33.901496 10851 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:33.901968 10851 webserver.cc:460] Webserver started at http://127.9.112.65:34967/ using document root <none> and password file <none>
I20260925 22:00:33.902179 10851 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:33.902230 10851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:33.902356 10851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:33.903496 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "da37b2de5cd14d77b0534187eb53befb"
format_stamp: "Formatted at 2026-09-25 22:00:33 on dist-test-slave-794d"
server_key: "a2d15c00a632d3fd1af7458b0083785a"
server_key_iv: "a1085ee46fb6d1994195d41f7de9d45d"
server_key_version: "encryptionkey@0"
I20260925 22:00:33.903861 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "da37b2de5cd14d77b0534187eb53befb"
format_stamp: "Formatted at 2026-09-25 22:00:33 on dist-test-slave-794d"
server_key: "a2d15c00a632d3fd1af7458b0083785a"
server_key_iv: "a1085ee46fb6d1994195d41f7de9d45d"
server_key_version: "encryptionkey@0"
I20260925 22:00:33.905246 10851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:33.905977 10866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:33.906127 10851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:00:33.906211 10851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "da37b2de5cd14d77b0534187eb53befb"
format_stamp: "Formatted at 2026-09-25 22:00:33 on dist-test-slave-794d"
server_key: "a2d15c00a632d3fd1af7458b0083785a"
server_key_iv: "a1085ee46fb6d1994195d41f7de9d45d"
server_key_version: "encryptionkey@0"
I20260925 22:00:33.906277 10851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:33.930856 10851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:33.933056 10851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:33.933207 10851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:33.933717 10851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:33.933760 10851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:33.933790 10851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:33.933808 10851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:33.940217 10851 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:44593
I20260925 22:00:33.940315 10981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:44593 every 8 connection(s)
Sep 25 22:00:33 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373633, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:33.940644 10851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:33.943550 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:33.934240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:35555 (local address 127.9.112.126:40981)
0925 22:00:33.934286 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:33.934288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:33.934533 (+ 245us) server_negotiation.cc:407] Connection header received
0925 22:00:33.935058 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:33.935059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:33.935074 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:33.935123 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:33.936073 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.936865 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.937848 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.938152 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.938235 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:33.940926 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:33.940928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:33.940929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:33.940937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:33.942264 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:33.942660 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:33.942661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:33.942662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:33.942714 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:33.942937 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:33.942938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:33.942939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:33.943053 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:33.943108 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:33.943374 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 22:00:33.943461 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:33.943720 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:33.934207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.9.112.65:35555)
0925 22:00:33.934408 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:33.934453 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:33.934968 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:33.935207 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:33.935212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:33.935548 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:33.935958 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:33.935966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.936958 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.936959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:33.937781 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:33.937783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.938257 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.938257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:33.938395 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:33.938952 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:33.938965 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:33.940828 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:33.942352 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:33.942358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:33.942360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:33.942592 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:33.942768 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:33.942768 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:33.942769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:33.942881 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:33.943106 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:33.943113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:33.943252 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 22:00:33.943345 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 22:00:33.943508 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260925 22:00:33.944242 10982 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40981
I20260925 22:00:33.944368 10982 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:33.944543 10982 heartbeater.cc:511] Master 127.9.112.126:40981 requested a full tablet report, sending...
I20260925 22:00:33.944612 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10851
I20260925 22:00:33.944710 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:33.944882 9665 external_mini_cluster.cc:1428] Setting key 88fb762a8c18f9d730dd6fa12aa95270
I20260925 22:00:33.945421 10778 ts_manager.cc:194] Registered new tserver with Master: da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593)
I20260925 22:00:33.946035 10778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:35555
I20260925 22:00:33.951426 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:33.945312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:55985 (local address 127.9.112.126:40981)
0925 22:00:33.946204 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:33.946206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:33.946214 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:33.946232 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:33.946232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:33.946241 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:33.946298 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:33.946682 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.947205 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.947859 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.948077 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.948134 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:33.949154 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:33.949155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:33.949155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:33.949161 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:33.950418 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:33.950784 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:33.950785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:33.950786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:33.950833 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:33.951015 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:33.951015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:33.951015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:33.951103 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:33.951153 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:33.951303 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 22:00:33.951360 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872}
I20260925 22:00:33.951479 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:33.945279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.9.112.65:55985)
0925 22:00:33.945412 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:33.945422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:33.945472 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:33.946317 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:33.946318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:33.946514 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:33.946581 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:33.946582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.947292 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.947293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:33.947789 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:33.947790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:33.948185 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:33.948186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:33.948260 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:33.948694 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:33.948698 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:33.949075 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:33.950498 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:33.950499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:33.950499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:33.950720 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:33.950892 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:33.950892 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:33.950893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:33.950956 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:33.951153 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:33.951154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:33.951203 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:00:33.951277 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:00:33.951379 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:33.972553 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40981
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:34.052565 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:34.052788 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:34.052811 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:34.053964 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:34.054003 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:34.054023 10991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:34.054039 10991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:34.055413 10991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:34.055612 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:34.055814 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:34.060833 10997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:34.060833 10996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:34.060865 10991 server_base.cc:1034] running on GCE node
W20260925 22:00:34.060837 10999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:34.061513 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:34.061792 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:34.062963 10991 hybrid_clock.cc:648] HybridClock initialized: now 1790373634062942 us; error 36 us; skew 500 ppm
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:34.065552 10991 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:34.065997 10991 webserver.cc:460] Webserver started at http://127.9.112.66:32973/ using document root <none> and password file <none>
I20260925 22:00:34.066200 10991 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:34.066255 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:34.066385 10991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:34.067412 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "8ef5071f65e84db89bd7c32052654112"
format_stamp: "Formatted at 2026-09-25 22:00:34 on dist-test-slave-794d"
server_key: "9f4b7863f9438007723e5d0aa3eead9f"
server_key_iv: "1868b55b78a9d542454030048623bc1a"
server_key_version: "encryptionkey@0"
I20260925 22:00:34.067757 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "8ef5071f65e84db89bd7c32052654112"
format_stamp: "Formatted at 2026-09-25 22:00:34 on dist-test-slave-794d"
server_key: "9f4b7863f9438007723e5d0aa3eead9f"
server_key_iv: "1868b55b78a9d542454030048623bc1a"
server_key_version: "encryptionkey@0"
I20260925 22:00:34.069228 10991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:34.069980 11006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.070163 10991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:34.070284 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "8ef5071f65e84db89bd7c32052654112"
format_stamp: "Formatted at 2026-09-25 22:00:34 on dist-test-slave-794d"
server_key: "9f4b7863f9438007723e5d0aa3eead9f"
server_key_iv: "1868b55b78a9d542454030048623bc1a"
server_key_version: "encryptionkey@0"
I20260925 22:00:34.070349 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:34.095836 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:34.097944 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:34.098083 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:34.098628 10991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:34.098670 10991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.098699 10991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:34.098716 10991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:34.105266 10991 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:36097
I20260925 22:00:34.105348 11121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:36097 every 8 connection(s)
I20260925 22:00:34.105633 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:34.107636 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 10991
I20260925 22:00:34.107718 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:34.107954 9665 external_mini_cluster.cc:1428] Setting key b5615249d369aa2d5814772089c487b5
I20260925 22:00:34.108398 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.099059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:56383 (local address 127.9.112.126:40981)
0925 22:00:34.099127 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.099129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.099359 (+ 230us) server_negotiation.cc:407] Connection header received
0925 22:00:34.100059 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.100061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.100072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.100178 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.101099 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.101899 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.102986 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.103235 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.103291 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.105573 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.105574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.105575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.105583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.106927 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.107293 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.107294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.107294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.107343 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.107557 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.107558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.107559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.107705 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.107773 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.108174 (+ 401us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.108271 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:00:34.108566 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.099044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.9.112.66:56383)
0925 22:00:34.099214 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.099264 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.099953 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.100293 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.100298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.100688 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:34.100998 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.101005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.101993 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.101994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.102901 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.102904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.103332 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.103333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.103461 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.103973 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:34.103984 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:34.105470 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:34.107002 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.107008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.107010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.107233 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.107392 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.107393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.107393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.107486 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.107803 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:34.107810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:34.107984 (+ 174us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.108115 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.108298 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260925 22:00:34.109170 11122 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40981
I20260925 22:00:34.109256 11122 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:34.109483 11122 heartbeater.cc:511] Master 127.9.112.126:40981 requested a full tablet report, sending...
I20260925 22:00:34.110036 10778 ts_manager.cc:194] Registered new tserver with Master: 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097)
I20260925 22:00:34.110504 10778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:56383
I20260925 22:00:34.116245 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.110168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:52263 (local address 127.9.112.126:40981)
0925 22:00:34.110210 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.110212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.110366 (+ 154us) server_negotiation.cc:407] Connection header received
0925 22:00:34.110412 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.110413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.110422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.110474 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.110814 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.111457 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.112473 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.112655 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.112705 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.113734 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.113735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.113736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.113742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.115094 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.115469 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.115470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.115470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.115535 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.115732 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.115733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.115733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.115870 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.115921 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.116084 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.116149 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:00:34.116271 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.110166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.9.112.66:52263)
0925 22:00:34.110290 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.110302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.110365 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.110478 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.110479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.110686 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:34.110758 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.110760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.111614 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.111616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.112393 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.112396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.112743 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.112744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.112819 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.113230 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:34.113235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:34.113622 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:34.115179 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.115181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.115181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.115389 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.115600 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.115601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.115601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.115675 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.115933 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:34.115934 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:34.115987 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.116060 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.116164 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:34.135185 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40981
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:34.216430 11131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:34.216661 11131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:34.216689 11131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:34.217963 11131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:34.218011 11131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:34.218032 11131 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:34.218050 11131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:34.219548 11131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:34.219786 11131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:34.220003 11131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:34.224967 11139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:34.225044 11131 server_base.cc:1034] running on GCE node
W20260925 22:00:34.224999 11136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:34.224990 11137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:34.225433 11131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:34.225720 11131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:34.226893 11131 hybrid_clock.cc:648] HybridClock initialized: now 1790373634226873 us; error 34 us; skew 500 ppm
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:34.229542 11131 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:34.229955 11131 webserver.cc:460] Webserver started at http://127.9.112.67:39559/ using document root <none> and password file <none>
I20260925 22:00:34.230135 11131 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:34.230175 11131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:34.230273 11131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:34.231266 11131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "0e0a1bab90484c2d99f83486e2a87dfe"
format_stamp: "Formatted at 2026-09-25 22:00:34 on dist-test-slave-794d"
server_key: "344ad069874c5ce8d5bec468c9d85bc1"
server_key_iv: "1282ae53190bcbb7a3984c842e8728b2"
server_key_version: "encryptionkey@0"
I20260925 22:00:34.231555 11131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "0e0a1bab90484c2d99f83486e2a87dfe"
format_stamp: "Formatted at 2026-09-25 22:00:34 on dist-test-slave-794d"
server_key: "344ad069874c5ce8d5bec468c9d85bc1"
server_key_iv: "1282ae53190bcbb7a3984c842e8728b2"
server_key_version: "encryptionkey@0"
I20260925 22:00:34.232821 11131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:34.233659 11146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.233904 11131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:34.233994 11131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "0e0a1bab90484c2d99f83486e2a87dfe"
format_stamp: "Formatted at 2026-09-25 22:00:34 on dist-test-slave-794d"
server_key: "344ad069874c5ce8d5bec468c9d85bc1"
server_key_iv: "1282ae53190bcbb7a3984c842e8728b2"
server_key_version: "encryptionkey@0"
I20260925 22:00:34.234058 11131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:34.245080 11131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:34.247069 11131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:34.247219 11131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:34.247730 11131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:34.247776 11131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.247807 11131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:34.247823 11131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.253926 11131 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:36447
I20260925 22:00:34.254051 11261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:36447 every 8 connection(s)
I20260925 22:00:34.254304 11131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:34.257707 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.248189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:43573 (local address 127.9.112.126:40981)
0925 22:00:34.248255 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.248258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.248530 (+ 272us) server_negotiation.cc:407] Connection header received
0925 22:00:34.249047 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.249050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.249062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.249145 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.250007 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.250778 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.252260 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.252578 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.252654 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.255160 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.255161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.255161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.255169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.256382 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.256793 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.256795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.256795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.256858 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.257075 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.257076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.257076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.257218 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.257273 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.257535 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.257617 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:00:34.257892 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.248163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.9.112.67:43573)
0925 22:00:34.248404 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.248445 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.248933 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.249247 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.249252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.249604 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:34.249903 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.249911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.250872 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.250874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.252149 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.252153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.252655 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.252656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.252797 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.253423 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:34.253435 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:34.255065 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:34.256448 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.256456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.256459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.256675 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.256912 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.256912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.256913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.257019 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.257273 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:34.257279 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:34.257416 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.257506 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.257663 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":75,"threads_started":1}
I20260925 22:00:34.258337 11262 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40981
I20260925 22:00:34.258414 11262 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:34.258585 11262 heartbeater.cc:511] Master 127.9.112.126:40981 requested a full tablet report, sending...
I20260925 22:00:34.258955 10779 ts_manager.cc:194] Registered new tserver with Master: 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447)
I20260925 22:00:34.259436 10779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:43573
I20260925 22:00:34.259910 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 11131
I20260925 22:00:34.259989 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:34.260146 9665 external_mini_cluster.cc:1428] Setting key 1e60fa43ad6676c2ff94ee42e3f271eb
I20260925 22:00:34.260864 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:34.265582 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.259252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.9.112.67:36639)
0925 22:00:34.259388 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.259398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.259445 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.259589 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.259590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.259812 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:34.259860 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.259862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.260654 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.260654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.261405 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.261407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.261807 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.261808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.261880 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.262403 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:34.262408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:34.262788 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:34.264487 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.264488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.264489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.264659 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.264879 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.264880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.264880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.264959 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.265267 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:34.265268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:34.265317 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.265402 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.265503 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":54,"threads_started":1}
I20260925 22:00:34.265617 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.259332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:36639 (local address 127.9.112.126:40981)
0925 22:00:34.259371 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.259373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.259437 (+ 64us) server_negotiation.cc:407] Connection header received
0925 22:00:34.259481 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.259482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.259491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.259545 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.259930 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.260591 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.261498 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.261735 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.261798 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.262877 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.262878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.262879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.262886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.264395 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.264724 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.264726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.264726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.264789 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.265031 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.265032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.265033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.265180 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.265250 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.265439 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.265520 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:34.284242 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.277515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.9.112.126:40981)
0925 22:00:34.277573 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.277575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.277582 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:00:34.277605 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.277606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.277614 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.277658 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.278252 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.278735 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.279454 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.279636 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.279692 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.281970 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.281972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.281972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.281981 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.283198 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.283519 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.283520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.283521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.283574 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.283732 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.283733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.283733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.283859 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.283946 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.284064 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.284144 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:00:34.285856 10779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:34.287415 10779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:34.293737 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.289831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44593 (local address 127.0.0.1:42258)
0925 22:00:34.290121 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.290130 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.290168 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.290450 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.290450 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.290452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.290481 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.290482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.291728 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.291729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.292810 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.292811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.293450 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.293451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.293525 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.293538 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.293602 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.293655 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":25,"thread_start_us":67,"threads_started":1}
I20260925 22:00:34.293761 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.289970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.9.112.65:44593)
0925 22:00:34.290263 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.290267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.290276 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:34.290310 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.290312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.290326 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.290395 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.290544 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.291642 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.292879 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.293373 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.293437 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.293497 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.293621 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.293656 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":75,"threads_started":1}
I20260925 22:00:34.293956 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.289729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.9.112.66:36097)
0925 22:00:34.289965 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.289969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.290087 (+ 118us) server_negotiation.cc:407] Connection header received
0925 22:00:34.290175 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.290177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.290206 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.290275 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.290501 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.291275 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.292691 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.293445 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.293513 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.293565 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.293817 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.293852 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":69,"threads_started":1}
I20260925 22:00:34.294042 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.289711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.9.112.67:36447)
0925 22:00:34.289880 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.289885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.290048 (+ 163us) server_negotiation.cc:407] Connection header received
0925 22:00:34.290176 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.290178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.290192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.290265 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.290520 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.291666 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.292735 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.293446 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.293513 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.293565 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.293918 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.293958 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260925 22:00:34.294147 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.289632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:36097 (local address 127.0.0.1:45854)
0925 22:00:34.290026 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.290037 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.290111 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.290341 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.290347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.290349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.290428 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.290430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.291359 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.291360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.292609 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.292611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.293599 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.293600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.293687 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.293689 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.293732 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.293774 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":10,"spinlock_wait_cycles":768,"thread_start_us":113,"threads_started":1}
I20260925 22:00:34.293852 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.289632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:36447 (local address 127.0.0.1:32800)
0925 22:00:34.289954 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.289995 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.290064 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.290284 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.290350 (+ 66us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.290353 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.290442 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.290445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.291747 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.291748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.292654 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.292656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.293599 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.293600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.293682 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.293686 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.293732 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.293774 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":26,"thread_start_us":52,"threads_started":1}
I20260925 22:00:34.294875 10916 tablet_service.cc:1467] Processing CreateTablet for tablet f25dd95e6fdd4f50a2552add7f440d94 (DEFAULT_TABLE table=test-table [id=9dfe1411c473482ea1f0cd9acc76bc47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:34.294845 11196 tablet_service.cc:1467] Processing CreateTablet for tablet f25dd95e6fdd4f50a2552add7f440d94 (DEFAULT_TABLE table=test-table [id=9dfe1411c473482ea1f0cd9acc76bc47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:34.294847 11056 tablet_service.cc:1467] Processing CreateTablet for tablet f25dd95e6fdd4f50a2552add7f440d94 (DEFAULT_TABLE table=test-table [id=9dfe1411c473482ea1f0cd9acc76bc47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:34.295190 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25dd95e6fdd4f50a2552add7f440d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.295197 11056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25dd95e6fdd4f50a2552add7f440d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.295197 11196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25dd95e6fdd4f50a2552add7f440d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.298231 11285 tablet_bootstrap.cc:492] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Bootstrap starting.
I20260925 22:00:34.298552 11286 tablet_bootstrap.cc:492] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Bootstrap starting.
I20260925 22:00:34.298614 11287 tablet_bootstrap.cc:492] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Bootstrap starting.
I20260925 22:00:34.299150 11287 tablet_bootstrap.cc:654] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.299156 11285 tablet_bootstrap.cc:654] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.299431 11287 log.cc:824] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:34.299472 11285 log.cc:824] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:34.299502 11286 tablet_bootstrap.cc:654] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.299793 11286 log.cc:824] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:34.300117 11287 tablet_bootstrap.cc:492] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: No bootstrap required, opened a new log
I20260925 22:00:34.300173 11287 ts_tablet_manager.cc:1397] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:34.300454 11286 tablet_bootstrap.cc:492] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: No bootstrap required, opened a new log
I20260925 22:00:34.300552 11286 ts_tablet_manager.cc:1397] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:34.300817 11285 tablet_bootstrap.cc:492] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: No bootstrap required, opened a new log
I20260925 22:00:34.300879 11285 ts_tablet_manager.cc:1397] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:00:34.302203 11286 raft_consensus.cc:348] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.302182 11287 raft_consensus.cc:348] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.302326 11286 raft_consensus.cc:374] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.302345 11286 raft_consensus.cc:729] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0a1bab90484c2d99f83486e2a87dfe, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.302337 11287 raft_consensus.cc:374] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.302378 11287 raft_consensus.cc:729] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef5071f65e84db89bd7c32052654112, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.302444 11286 consensus_queue.cc:260] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.302510 11287 consensus_queue.cc:260] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.302683 11286 ts_tablet_manager.cc:1428] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:34.302769 11287 ts_tablet_manager.cc:1428] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:34.302788 11285 raft_consensus.cc:348] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.302894 11122 heartbeater.cc:503] Master 127.9.112.126:40981 was elected leader, sending a full tablet report...
I20260925 22:00:34.302922 11285 raft_consensus.cc:374] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.302954 11285 raft_consensus.cc:729] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da37b2de5cd14d77b0534187eb53befb, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.302956 11262 heartbeater.cc:503] Master 127.9.112.126:40981 was elected leader, sending a full tablet report...
I20260925 22:00:34.303054 11285 consensus_queue.cc:260] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.303318 10982 heartbeater.cc:503] Master 127.9.112.126:40981 was elected leader, sending a full tablet report...
I20260925 22:00:34.303428 11285 ts_tablet_manager.cc:1428] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260925 22:00:34.356290 11123 tablet.cc:2369] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:34.427603 11291 raft_consensus.cc:482] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:34.427752 11291 raft_consensus.cc:504] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.428058 11291 leader_election.cc:288] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097), da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593)
I20260925 22:00:34.432030 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.428481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:40781 (local address 127.9.112.65:44593)
0925 22:00:34.428533 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.428535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.428650 (+ 115us) server_negotiation.cc:407] Connection header received
0925 22:00:34.428721 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.428722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.428732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.428776 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.428932 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.429610 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.431083 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.431637 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.431714 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.431722 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.431926 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.431962 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:00:34.432030 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.428420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:44909 (local address 127.9.112.66:36097)
0925 22:00:34.428467 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.428469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.428480 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:00:34.428510 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.428511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.428521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.428616 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.428810 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.429583 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.431083 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.431644 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.431714 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.431722 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.431920 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.431958 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:00:34.432044 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.428274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:36097 (local address 127.9.112.67:44909)
0925 22:00:34.428326 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.428361 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.428408 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.428634 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.428635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.428637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.428712 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.428732 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.429700 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.429702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.430997 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.430998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.431745 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.431746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.431841 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.431843 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.431907 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.431942 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":48}
I20260925 22:00:34.432049 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.428404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44593 (local address 127.9.112.67:40781)
0925 22:00:34.428597 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.428607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.428661 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.428803 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.428804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.428805 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.428863 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.428864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.429700 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.429702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.430993 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.430996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.431745 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.431746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.431838 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.431843 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.431901 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.431941 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":33,"thread_start_us":122,"threads_started":1}
I20260925 22:00:34.432493 11076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25dd95e6fdd4f50a2552add7f440d94" candidate_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef5071f65e84db89bd7c32052654112" is_pre_election: true
I20260925 22:00:34.432534 10936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25dd95e6fdd4f50a2552add7f440d94" candidate_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da37b2de5cd14d77b0534187eb53befb" is_pre_election: true
I20260925 22:00:34.432668 10936 raft_consensus.cc:2454] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e0a1bab90484c2d99f83486e2a87dfe in term 0.
I20260925 22:00:34.432627 11076 raft_consensus.cc:2454] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e0a1bab90484c2d99f83486e2a87dfe in term 0.
I20260925 22:00:34.432880 11149 leader_election.cc:302] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0a1bab90484c2d99f83486e2a87dfe, da37b2de5cd14d77b0534187eb53befb; no voters:
I20260925 22:00:34.432991 11291 raft_consensus.cc:2781] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:34.433049 11291 raft_consensus.cc:482] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:34.433064 11291 raft_consensus.cc:3037] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.433980 11291 raft_consensus.cc:504] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.434111 11291 leader_election.cc:288] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [CANDIDATE]: Term 1 election: Requested vote from peers 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097), da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593)
I20260925 22:00:34.434298 10936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25dd95e6fdd4f50a2552add7f440d94" candidate_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da37b2de5cd14d77b0534187eb53befb"
I20260925 22:00:34.434298 11076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25dd95e6fdd4f50a2552add7f440d94" candidate_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef5071f65e84db89bd7c32052654112"
I20260925 22:00:34.434382 10936 raft_consensus.cc:3037] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.434382 11076 raft_consensus.cc:3037] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.435197 11076 raft_consensus.cc:2454] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e0a1bab90484c2d99f83486e2a87dfe in term 1.
I20260925 22:00:34.435197 10936 raft_consensus.cc:2454] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e0a1bab90484c2d99f83486e2a87dfe in term 1.
I20260925 22:00:34.435391 11149 leader_election.cc:302] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0a1bab90484c2d99f83486e2a87dfe, da37b2de5cd14d77b0534187eb53befb; no voters:
I20260925 22:00:34.435497 11291 raft_consensus.cc:2781] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:34.435688 11291 raft_consensus.cc:686] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 LEADER]: Becoming Leader. State: Replica: 0e0a1bab90484c2d99f83486e2a87dfe, State: Running, Role: LEADER
I20260925 22:00:34.435818 11291 consensus_queue.cc:237] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } }
I20260925 22:00:34.436688 10779 catalog_manager.cc:5495] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67). New cstate: current_term: 1 leader_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } health_report { overall_health: UNKNOWN } } }
W20260925 22:00:34.441502 10983 tablet.cc:2369] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:34.476251 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.473610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.9.112.67:36447)
0925 22:00:34.473660 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.473663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.473723 (+ 60us) server_negotiation.cc:407] Connection header received
0925 22:00:34.473837 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.473838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.473858 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.473913 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:34.474055 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.474605 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.475393 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.475597 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.475670 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.475873 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.475981 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.476051 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.476096 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.476121 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:00:34.478084 10936 raft_consensus.cc:1261] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Refusing update from remote peer 0e0a1bab90484c2d99f83486e2a87dfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:34.478084 11076 raft_consensus.cc:1261] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Refusing update from remote peer 0e0a1bab90484c2d99f83486e2a87dfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:34.478447 11291 consensus_queue.cc:1035] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:34.478593 11295 consensus_queue.cc:1035] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:34.481110 11299 mvcc.cc:205] Tried to move back new op lower bound from 7333370406819602432 to 7333370406649786368. Current Snapshot: MvccSnapshot[applied={T|T < 7333370406819602432}]
I20260925 22:00:34.481206 11300 mvcc.cc:205] Tried to move back new op lower bound from 7333370406819602432 to 7333370406649786368. Current Snapshot: MvccSnapshot[applied={T|T < 7333370406819602432}]
I20260925 22:00:34.484155 10778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:34.484263 10778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:34.485122 10778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: Sending DeleteTablet for 3 replicas of tablet f25dd95e6fdd4f50a2552add7f440d94
I20260925 22:00:34.485450 11196 tablet_service.cc:1514] Processing DeleteTablet for tablet f25dd95e6fdd4f50a2552add7f440d94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:34 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:32800
I20260925 22:00:34.485507 11056 tablet_service.cc:1514] Processing DeleteTablet for tablet f25dd95e6fdd4f50a2552add7f440d94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:34 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:45854
I20260925 22:00:34.485493 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet f25dd95e6fdd4f50a2552add7f440d94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:34 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:42258
I20260925 22:00:34.485735 11307 tablet_replica.cc:330] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: stopping tablet replica
I20260925 22:00:34.485867 11307 raft_consensus.cc:2229] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:34.485903 11308 tablet_replica.cc:330] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: stopping tablet replica
I20260925 22:00:34.486034 11308 raft_consensus.cc:2229] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:34.485984 10778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:34.486122 10778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:34.486258 11308 raft_consensus.cc:2258] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:34.486289 11307 raft_consensus.cc:2258] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:34.486672 11309 tablet_replica.cc:330] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: stopping tablet replica
I20260925 22:00:34.486874 11309 raft_consensus.cc:2229] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:34.486937 11307 ts_tablet_manager.cc:1905] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:34.487048 11309 raft_consensus.cc:2258] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:34.487577 11309 ts_tablet_manager.cc:1905] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:34.487624 11196 tablet_service.cc:1467] Processing CreateTablet for tablet 201ec970f9414f1a8d2f2c656416ae2e (DEFAULT_TABLE table=test-table [id=eb12b00bde674b228b681b77f10b3d42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:34.487730 11196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 201ec970f9414f1a8d2f2c656416ae2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.488708 11307 ts_tablet_manager.cc:1918] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:34.488773 11307 log.cc:1196] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/f25dd95e6fdd4f50a2552add7f440d94
I20260925 22:00:34.488970 11286 tablet_bootstrap.cc:492] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Bootstrap starting.
I20260925 22:00:34.489027 11307 ts_tablet_manager.cc:1939] T f25dd95e6fdd4f50a2552add7f440d94 P 0e0a1bab90484c2d99f83486e2a87dfe: Deleting consensus metadata
I20260925 22:00:34.489403 11286 tablet_bootstrap.cc:654] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.489548 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 201ec970f9414f1a8d2f2c656416ae2e (DEFAULT_TABLE table=test-table [id=eb12b00bde674b228b681b77f10b3d42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:34.489655 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 201ec970f9414f1a8d2f2c656416ae2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.489658 10763 catalog_manager.cc:4841] TS 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447): tablet f25dd95e6fdd4f50a2552add7f440d94 (table test-table [id=9dfe1411c473482ea1f0cd9acc76bc47]) successfully deleted
I20260925 22:00:34.490017 11309 ts_tablet_manager.cc:1918] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:34.490075 11309 log.cc:1196] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/f25dd95e6fdd4f50a2552add7f440d94
I20260925 22:00:34.490291 11309 ts_tablet_manager.cc:1939] T f25dd95e6fdd4f50a2552add7f440d94 P 8ef5071f65e84db89bd7c32052654112: Deleting consensus metadata
I20260925 22:00:34.490948 11285 tablet_bootstrap.cc:492] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Bootstrap starting.
I20260925 22:00:34.491030 10765 catalog_manager.cc:4841] TS 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097): tablet f25dd95e6fdd4f50a2552add7f440d94 (table test-table [id=9dfe1411c473482ea1f0cd9acc76bc47]) successfully deleted
I20260925 22:00:34.491271 11056 tablet_service.cc:1467] Processing CreateTablet for tablet 201ec970f9414f1a8d2f2c656416ae2e (DEFAULT_TABLE table=test-table [id=eb12b00bde674b228b681b77f10b3d42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:34.491331 11285 tablet_bootstrap.cc:654] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.491355 11056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 201ec970f9414f1a8d2f2c656416ae2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.491961 11285 tablet_bootstrap.cc:492] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: No bootstrap required, opened a new log
I20260925 22:00:34.492017 11285 ts_tablet_manager.cc:1397] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:34.492163 11285 raft_consensus.cc:348] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.492238 11285 raft_consensus.cc:374] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.492280 11285 raft_consensus.cc:729] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da37b2de5cd14d77b0534187eb53befb, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.492269 11287 tablet_bootstrap.cc:492] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Bootstrap starting.
I20260925 22:00:34.492324 11285 consensus_queue.cc:260] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.492460 11285 ts_tablet_manager.cc:1428] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:34.492658 11287 tablet_bootstrap.cc:654] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.493249 11287 tablet_bootstrap.cc:492] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: No bootstrap required, opened a new log
I20260925 22:00:34.493238 11308 ts_tablet_manager.cc:1905] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:34.493292 11287 ts_tablet_manager.cc:1397] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:34.493432 11287 raft_consensus.cc:348] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.493491 11287 raft_consensus.cc:374] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.493508 11287 raft_consensus.cc:729] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef5071f65e84db89bd7c32052654112, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.493553 11287 consensus_queue.cc:260] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.493644 11287 ts_tablet_manager.cc:1428] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.495011 11308 ts_tablet_manager.cc:1918] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:34.495085 11308 log.cc:1196] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/f25dd95e6fdd4f50a2552add7f440d94
I20260925 22:00:34.495074 11286 tablet_bootstrap.cc:492] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: No bootstrap required, opened a new log
I20260925 22:00:34.495121 11286 ts_tablet_manager.cc:1397] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 22:00:34.495349 11308 ts_tablet_manager.cc:1939] T f25dd95e6fdd4f50a2552add7f440d94 P da37b2de5cd14d77b0534187eb53befb: Deleting consensus metadata
I20260925 22:00:34.495292 11286 raft_consensus.cc:348] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.495384 11286 raft_consensus.cc:374] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.495405 11286 raft_consensus.cc:729] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0a1bab90484c2d99f83486e2a87dfe, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.495455 11286 consensus_queue.cc:260] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.495544 11286 ts_tablet_manager.cc:1428] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:34.495895 10765 catalog_manager.cc:4841] TS da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593): tablet f25dd95e6fdd4f50a2552add7f440d94 (table test-table [id=9dfe1411c473482ea1f0cd9acc76bc47]) successfully deleted
W20260925 22:00:34.504859 11263 tablet.cc:2369] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:34.716423 11292 raft_consensus.cc:482] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:34.716524 11292 raft_consensus.cc:504] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.716861 11292 leader_election.cc:288] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593), 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447)
I20260925 22:00:34.720031 11314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.717056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44593 (local address 127.9.112.66:39581)
0925 22:00:34.717178 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.717187 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.717251 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.717375 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.717376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.717378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.717451 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.717453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.718051 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.718052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.719021 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.719024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.719729 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.719730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.719802 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.719807 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.719893 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.719941 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":43,"thread_start_us":69,"threads_started":1}
I20260925 22:00:34.720041 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.717126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:39581 (local address 127.9.112.65:44593)
0925 22:00:34.717175 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.717177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.717289 (+ 112us) server_negotiation.cc:407] Connection header received
0925 22:00:34.717316 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.717316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.717324 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.717367 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.717507 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.717974 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.719117 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.719641 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.719716 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.719725 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.719925 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.719969 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:00:34.720103 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.717144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:40495 (local address 127.9.112.67:36447)
0925 22:00:34.717193 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.717195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.717246 (+ 51us) server_negotiation.cc:407] Connection header received
0925 22:00:34.717319 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.717320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.717332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.717379 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.717530 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.718027 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.719236 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.719779 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.719836 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.719844 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.720002 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.720044 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:00:34.720283 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.717048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:36447 (local address 127.9.112.66:40495)
0925 22:00:34.717176 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.717186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.717256 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.717390 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.717391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.717392 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.717465 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.717467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.718106 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.718107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.719133 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.719136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.719848 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.719849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.719919 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.719923 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.720168 (+ 245us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.720213 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":18,"thread_start_us":63,"threads_started":1}
I20260925 22:00:34.720528 10936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "201ec970f9414f1a8d2f2c656416ae2e" candidate_uuid: "8ef5071f65e84db89bd7c32052654112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da37b2de5cd14d77b0534187eb53befb" is_pre_election: true
I20260925 22:00:34.720624 10936 raft_consensus.cc:2454] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef5071f65e84db89bd7c32052654112 in term 0.
I20260925 22:00:34.720813 11009 leader_election.cc:302] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef5071f65e84db89bd7c32052654112, da37b2de5cd14d77b0534187eb53befb; no voters:
I20260925 22:00:34.720803 11216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "201ec970f9414f1a8d2f2c656416ae2e" candidate_uuid: "8ef5071f65e84db89bd7c32052654112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" is_pre_election: true
I20260925 22:00:34.720927 11292 raft_consensus.cc:2781] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:34.720927 11216 raft_consensus.cc:2454] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef5071f65e84db89bd7c32052654112 in term 0.
I20260925 22:00:34.720960 11292 raft_consensus.cc:482] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:34.720976 11292 raft_consensus.cc:3037] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.721638 11292 raft_consensus.cc:504] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.721771 11292 leader_election.cc:288] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [CANDIDATE]: Term 1 election: Requested vote from peers da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593), 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447)
I20260925 22:00:34.721942 10936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "201ec970f9414f1a8d2f2c656416ae2e" candidate_uuid: "8ef5071f65e84db89bd7c32052654112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da37b2de5cd14d77b0534187eb53befb"
I20260925 22:00:34.721959 11216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "201ec970f9414f1a8d2f2c656416ae2e" candidate_uuid: "8ef5071f65e84db89bd7c32052654112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0a1bab90484c2d99f83486e2a87dfe"
I20260925 22:00:34.722018 11216 raft_consensus.cc:3037] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.722012 10936 raft_consensus.cc:3037] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.722599 10936 raft_consensus.cc:2454] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef5071f65e84db89bd7c32052654112 in term 1.
I20260925 22:00:34.722648 11216 raft_consensus.cc:2454] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef5071f65e84db89bd7c32052654112 in term 1.
I20260925 22:00:34.722787 11007 leader_election.cc:302] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0a1bab90484c2d99f83486e2a87dfe, 8ef5071f65e84db89bd7c32052654112; no voters:
I20260925 22:00:34.722903 11292 raft_consensus.cc:2781] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:34.723026 11292 raft_consensus.cc:686] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef5071f65e84db89bd7c32052654112, State: Running, Role: LEADER
I20260925 22:00:34.723089 11292 consensus_queue.cc:237] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.723852 10778 catalog_manager.cc:5495] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef5071f65e84db89bd7c32052654112 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "8ef5071f65e84db89bd7c32052654112" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } health_report { overall_health: UNKNOWN } } }
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373633, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:34.760208 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.753251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40981 (local address 127.0.0.1:49218)
0925 22:00:34.753397 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.753407 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.753471 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.753592 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.753593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.753908 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:34.753955 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.753957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.754661 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.754662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.755275 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.755277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.755625 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.755626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.755671 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.756245 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:34.756249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:34.757641 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:34.759164 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.759172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.759175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.759402 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.759635 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:34.759635 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:34.759635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:34.759692 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:34.759920 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:34.759928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:34.759974 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.760035 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.760131 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260925 22:00:34.760293 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.753323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.9.112.126:40981)
0925 22:00:34.753417 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.753419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.753466 (+ 47us) server_negotiation.cc:407] Connection header received
0925 22:00:34.753519 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.753520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.753528 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.753578 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.754029 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.754538 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.755352 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.755552 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.755605 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.757791 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.757792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.757792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.757799 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.759082 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.759491 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.759492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.759493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.759564 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.759731 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.759732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.759732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.759849 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.759904 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.760064 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.760190 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":70}
I20260925 22:00:34.762282 10778 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49218:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 22:00:34.762501 10778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:34.763957 11196 tablet_service.cc:1467] Processing CreateTablet for tablet 078e73d3377c4706bc0c822581985716 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52c190c957d24f78ba5abd8ba57baf58]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:00:34.763990 11056 tablet_service.cc:1467] Processing CreateTablet for tablet 078e73d3377c4706bc0c822581985716 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52c190c957d24f78ba5abd8ba57baf58]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:00:34.764001 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 078e73d3377c4706bc0c822581985716 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52c190c957d24f78ba5abd8ba57baf58]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:00:34.764135 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078e73d3377c4706bc0c822581985716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.764135 11196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078e73d3377c4706bc0c822581985716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.764134 11056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078e73d3377c4706bc0c822581985716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:34.765355 11285 tablet_bootstrap.cc:492] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: Bootstrap starting.
I20260925 22:00:34.765596 11287 tablet_bootstrap.cc:492] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: Bootstrap starting.
I20260925 22:00:34.765777 11285 tablet_bootstrap.cc:654] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.765977 11287 tablet_bootstrap.cc:654] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.766294 11286 tablet_bootstrap.cc:492] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: Bootstrap starting.
I20260925 22:00:34.766425 11285 tablet_bootstrap.cc:492] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: No bootstrap required, opened a new log
I20260925 22:00:34.766469 11285 ts_tablet_manager.cc:1397] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:34.766583 11287 tablet_bootstrap.cc:492] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: No bootstrap required, opened a new log
I20260925 22:00:34.766595 11286 tablet_bootstrap.cc:654] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:34.766619 11287 ts_tablet_manager.cc:1397] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:34.766609 11285 raft_consensus.cc:348] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.766683 11285 raft_consensus.cc:374] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.766702 11285 raft_consensus.cc:729] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da37b2de5cd14d77b0534187eb53befb, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.766750 11287 raft_consensus.cc:348] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.766824 11287 raft_consensus.cc:374] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.766842 11287 raft_consensus.cc:729] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef5071f65e84db89bd7c32052654112, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.766810 11285 consensus_queue.cc:260] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.766876 11287 consensus_queue.cc:260] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.766912 11285 ts_tablet_manager.cc:1428] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.766949 11287 ts_tablet_manager.cc:1428] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.767076 11286 tablet_bootstrap.cc:492] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: No bootstrap required, opened a new log
I20260925 22:00:34.767127 11286 ts_tablet_manager.cc:1397] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 22:00:34.767131 11293 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.767226 11293 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.767230 11286 raft_consensus.cc:348] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.767303 11286 raft_consensus.cc:374] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:34.767338 11286 raft_consensus.cc:729] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0a1bab90484c2d99f83486e2a87dfe, State: Initialized, Role: FOLLOWER
I20260925 22:00:34.767385 11286 consensus_queue.cc:260] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.767369 11292 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.767442 11292 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.767469 11286 ts_tablet_manager.cc:1428] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:34.767550 11291 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.767618 11291 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.855517 11293 raft_consensus.cc:482] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:34.855633 11293 raft_consensus.cc:504] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.856014 11293 leader_election.cc:288] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097), 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447)
I20260925 22:00:34.860100 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.856288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:47061 (local address 127.9.112.66:36097)
0925 22:00:34.856374 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.856375 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.856521 (+ 146us) server_negotiation.cc:407] Connection header received
0925 22:00:34.856578 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.856579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.856589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.856719 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.856848 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.857565 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.859169 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.859723 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.859796 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.859806 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.859988 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.860026 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:00:34.860100 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.856303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:45359 (local address 127.9.112.67:36447)
0925 22:00:34.856371 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.856374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.856519 (+ 145us) server_negotiation.cc:407] Connection header received
0925 22:00:34.856578 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.856579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.856589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.856669 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:34.856989 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.857700 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.859162 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.859739 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.859799 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.859806 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.859984 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.860026 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:00:34.860128 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.856222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:36097 (local address 127.9.112.65:47061)
0925 22:00:34.856439 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.856450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.856522 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.856680 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.856681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.856683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.856785 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.856788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.857649 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.857650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.859066 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.859071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.859825 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.859826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.859910 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.859913 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.859980 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.860008 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":32,"thread_start_us":101,"threads_started":1}
I20260925 22:00:34.860136 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.856222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:36447 (local address 127.9.112.65:45359)
0925 22:00:34.856439 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.856450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.856522 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.856680 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.856681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.856684 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:34.856785 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.856787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.857774 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.857775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.859066 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.859071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.859832 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.859833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.859902 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.859907 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.859963 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.860008 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":53,"thread_start_us":102,"threads_started":1}
I20260925 22:00:34.860416 11076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e73d3377c4706bc0c822581985716" candidate_uuid: "da37b2de5cd14d77b0534187eb53befb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef5071f65e84db89bd7c32052654112" is_pre_election: true
I20260925 22:00:34.860512 11076 raft_consensus.cc:2454] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da37b2de5cd14d77b0534187eb53befb in term 0.
I20260925 22:00:34.860574 11216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e73d3377c4706bc0c822581985716" candidate_uuid: "da37b2de5cd14d77b0534187eb53befb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" is_pre_election: true
I20260925 22:00:34.860661 11216 raft_consensus.cc:2454] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da37b2de5cd14d77b0534187eb53befb in term 0.
I20260925 22:00:34.860694 10869 leader_election.cc:302] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef5071f65e84db89bd7c32052654112, da37b2de5cd14d77b0534187eb53befb; no voters:
I20260925 22:00:34.860831 11293 raft_consensus.cc:2781] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:34.860883 11293 raft_consensus.cc:482] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:34.860898 11293 raft_consensus.cc:3037] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.861583 11293 raft_consensus.cc:504] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.861709 11293 leader_election.cc:288] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [CANDIDATE]: Term 1 election: Requested vote from peers 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097), 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447)
I20260925 22:00:34.861877 11216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e73d3377c4706bc0c822581985716" candidate_uuid: "da37b2de5cd14d77b0534187eb53befb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0a1bab90484c2d99f83486e2a87dfe"
I20260925 22:00:34.861919 11076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078e73d3377c4706bc0c822581985716" candidate_uuid: "da37b2de5cd14d77b0534187eb53befb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef5071f65e84db89bd7c32052654112"
I20260925 22:00:34.861960 11216 raft_consensus.cc:3037] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.861976 11076 raft_consensus.cc:3037] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:34.862684 11076 raft_consensus.cc:2454] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da37b2de5cd14d77b0534187eb53befb in term 1.
I20260925 22:00:34.862684 11216 raft_consensus.cc:2454] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da37b2de5cd14d77b0534187eb53befb in term 1.
I20260925 22:00:34.862843 10869 leader_election.cc:302] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef5071f65e84db89bd7c32052654112, da37b2de5cd14d77b0534187eb53befb; no voters:
I20260925 22:00:34.862926 11293 raft_consensus.cc:2781] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:34.863056 11293 raft_consensus.cc:686] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [term 1 LEADER]: Becoming Leader. State: Replica: da37b2de5cd14d77b0534187eb53befb, State: Running, Role: LEADER
I20260925 22:00:34.863116 11293 consensus_queue.cc:237] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } }
I20260925 22:00:34.863474 11328 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: TxnStatusTablet state changed. Reason: New leader da37b2de5cd14d77b0534187eb53befb. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.863585 11328 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:00:34.863834 11330 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 22:00:34.863850 10779 catalog_manager.cc:5495] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb reported cstate change: term changed from 0 to 1, leader changed from <none> to da37b2de5cd14d77b0534187eb53befb (127.9.112.65). New cstate: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:34.863945 11330 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 22:00:34.886250 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.883526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.9.112.65:44593)
0925 22:00:34.883580 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.883582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.883590 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:34.883665 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.883666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.883688 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.883742 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:34.883864 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.884334 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.885504 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.885696 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.885760 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.885950 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.886050 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.886103 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.886165 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.886187 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:00:34.886307 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.883446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44593 (local address 127.0.0.1:42266)
0925 22:00:34.883529 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.883539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.883583 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.883754 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.883755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.883756 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:34.883809 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.883811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.884436 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.884438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.885384 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.885388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.885772 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.885773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.885835 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.885855 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.886107 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.886112 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.886162 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.886220 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 22:00:34.887404 11076 raft_consensus.cc:1261] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Refusing update from remote peer da37b2de5cd14d77b0534187eb53befb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:34.887457 11216 raft_consensus.cc:1261] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Refusing update from remote peer da37b2de5cd14d77b0534187eb53befb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:34.887799 11293 consensus_queue.cc:1035] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:34.888001 11293 consensus_queue.cc:1035] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:34.888537 11292 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: TxnStatusTablet state changed. Reason: New leader da37b2de5cd14d77b0534187eb53befb. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.888553 11291 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: TxnStatusTablet state changed. Reason: New leader da37b2de5cd14d77b0534187eb53befb. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.888623 11292 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.888633 11291 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.889735 11298 mvcc.cc:205] Tried to move back new op lower bound from 7333370408496635904 to 7333370408399900672. Current Snapshot: MvccSnapshot[applied={T|T < 7333370408496635904}]
I20260925 22:00:34.889715 11335 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.889802 11335 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:00:34.889854 11335 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.889889 11335 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P da37b2de5cd14d77b0534187eb53befb: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:00:34.889977 11291 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.890045 11291 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P 0e0a1bab90484c2d99f83486e2a87dfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.890445 11292 tablet_replica.cc:439] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da37b2de5cd14d77b0534187eb53befb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 } } peers { permanent_uuid: "8ef5071f65e84db89bd7c32052654112" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36097 } } peers { permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 } } }
I20260925 22:00:34.890527 11292 tablet_replica.cc:442] T 078e73d3377c4706bc0c822581985716 P 8ef5071f65e84db89bd7c32052654112: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:34.894248 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.891991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.9.112.66:36097)
0925 22:00:34.892029 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.892030 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.892039 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:34.892123 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.892124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.892144 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.892230 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:34.892375 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.892845 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.893532 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.893748 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.893806 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.893949 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.894058 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.894118 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.894161 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.894179 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:34.899188 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.896930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:60225 (local address 127.9.112.65:44593)
0925 22:00:34.896992 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.896994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.897084 (+ 90us) server_negotiation.cc:407] Connection header received
0925 22:00:34.897145 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.897147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.897156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.897200 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:34.897314 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.897779 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.898533 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.898731 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.898787 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.898952 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.898993 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.899037 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.899088 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.899107 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:00:34.899394 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.896837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44593 (local address 127.9.112.66:60225)
0925 22:00:34.897016 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.897027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.897088 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.897201 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.897202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.897204 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:34.897262 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.897263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.897846 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.897847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.898452 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.898455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.898810 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.898810 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.898861 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.898875 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.899044 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.899046 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.899085 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.899126 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260925 22:00:34.901695 10936 raft_consensus.cc:1261] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Refusing update from remote peer 8ef5071f65e84db89bd7c32052654112: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:34.901695 11216 raft_consensus.cc:1261] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Refusing update from remote peer 8ef5071f65e84db89bd7c32052654112: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:34.901971 11292 consensus_queue.cc:1035] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da37b2de5cd14d77b0534187eb53befb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:34.902156 11315 consensus_queue.cc:1035] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0a1bab90484c2d99f83486e2a87dfe" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:34.912099 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.908899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:55497 (local address 127.9.112.66:36097)
0925 22:00:34.908942 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.908945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.909032 (+ 87us) server_negotiation.cc:407] Connection header received
0925 22:00:34.909101 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.909102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.909110 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.909144 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:34.909311 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.910013 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.911459 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.911646 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.911702 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.911893 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.911921 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.911955 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.912023 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.912041 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:00:34.912168 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.908815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:36097 (local address 127.9.112.65:55497)
0925 22:00:34.908973 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:34.908985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:34.909045 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:34.909150 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:34.909151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:34.909152 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:34.909223 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.909224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.910223 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.910225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.911370 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:34.911374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.911716 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.911717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:34.911784 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.911806 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:34.911971 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:34.911973 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:00:34.912022 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:00:34.912068 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260925 22:00:34.922026 10778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:34.922159 10778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:34.922851 10778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bbb0a52ca24476aa0fdbcd4cd310530: Sending DeleteTablet for 3 replicas of tablet 201ec970f9414f1a8d2f2c656416ae2e
I20260925 22:00:34.923120 11196 tablet_service.cc:1514] Processing DeleteTablet for tablet 201ec970f9414f1a8d2f2c656416ae2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:34 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:32800
I20260925 22:00:34.923225 11307 tablet_replica.cc:330] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: stopping tablet replica
I20260925 22:00:34.923233 11056 tablet_service.cc:1514] Processing DeleteTablet for tablet 201ec970f9414f1a8d2f2c656416ae2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:34 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:45854
I20260925 22:00:34.923281 11307 raft_consensus.cc:2229] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:34.923332 11307 raft_consensus.cc:2258] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:34.923332 11309 tablet_replica.cc:330] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: stopping tablet replica
I20260925 22:00:34.923383 11309 raft_consensus.cc:2229] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:34.923417 10916 tablet_service.cc:1514] Processing DeleteTablet for tablet 201ec970f9414f1a8d2f2c656416ae2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:34 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:42258
I20260925 22:00:34.923473 11309 raft_consensus.cc:2258] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:34.923516 11308 tablet_replica.cc:330] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: stopping tablet replica
I20260925 22:00:34.923579 11308 raft_consensus.cc:2229] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:34.923621 11308 raft_consensus.cc:2258] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:34.923902 11307 ts_tablet_manager.cc:1905] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:34.924399 11308 ts_tablet_manager.cc:1905] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:34.925532 11309 ts_tablet_manager.cc:1905] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:34.925956 11308 ts_tablet_manager.cc:1918] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:34.926009 11308 log.cc:1196] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/201ec970f9414f1a8d2f2c656416ae2e
I20260925 22:00:34.926230 11308 ts_tablet_manager.cc:1939] T 201ec970f9414f1a8d2f2c656416ae2e P da37b2de5cd14d77b0534187eb53befb: Deleting consensus metadata
I20260925 22:00:34.926595 11307 ts_tablet_manager.cc:1918] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:34.926653 11307 log.cc:1196] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/201ec970f9414f1a8d2f2c656416ae2e
I20260925 22:00:34.926846 11307 ts_tablet_manager.cc:1939] T 201ec970f9414f1a8d2f2c656416ae2e P 0e0a1bab90484c2d99f83486e2a87dfe: Deleting consensus metadata
I20260925 22:00:34.926975 11309 ts_tablet_manager.cc:1918] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:34.927022 11309 log.cc:1196] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/201ec970f9414f1a8d2f2c656416ae2e
I20260925 22:00:34.927199 11309 ts_tablet_manager.cc:1939] T 201ec970f9414f1a8d2f2c656416ae2e P 8ef5071f65e84db89bd7c32052654112: Deleting consensus metadata
I20260925 22:00:34.927208 10763 catalog_manager.cc:4841] TS 0e0a1bab90484c2d99f83486e2a87dfe (127.9.112.67:36447): tablet 201ec970f9414f1a8d2f2c656416ae2e (table test-table [id=eb12b00bde674b228b681b77f10b3d42]) successfully deleted
I20260925 22:00:34.927767 10765 catalog_manager.cc:4841] TS da37b2de5cd14d77b0534187eb53befb (127.9.112.65:44593): tablet 201ec970f9414f1a8d2f2c656416ae2e (table test-table [id=eb12b00bde674b228b681b77f10b3d42]) successfully deleted
I20260925 22:00:34.927840 10765 catalog_manager.cc:4841] TS 8ef5071f65e84db89bd7c32052654112 (127.9.112.66:36097): tablet 201ec970f9414f1a8d2f2c656416ae2e (table test-table [id=eb12b00bde674b228b681b77f10b3d42]) successfully deleted
Sep 25 22:00:34 dist-test-slave-794d krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:34.938776 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.924232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.9.112.65:44593)
0925 22:00:34.924270 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.924272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.928555 (+ 4283us) server_negotiation.cc:407] Connection header received
0925 22:00:34.928654 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.928655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.928665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.928693 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.929493 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.930231 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.931270 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.931555 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.931594 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.935410 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.935418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.935419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.935428 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.937429 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.937905 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.937910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.937910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.937984 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.938197 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.938198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.938198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.938383 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.938548 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.938593 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.938678 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 22:00:34.939059 10976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42268
I20260925 22:00:34.948385 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:34.940593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.9.112.126:40981)
0925 22:00:34.940640 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:34.940642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:34.940817 (+ 175us) server_negotiation.cc:407] Connection header received
0925 22:00:34.940909 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:34.940910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:34.940921 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:34.940950 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:34.941623 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.942458 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.943613 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:34.943920 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:34.943962 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:34.945508 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:34.945510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:34.945510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:34.945518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:34.947125 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.947530 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.947532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.947532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.947599 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:34.947798 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:34.947799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:34.947799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:34.947956 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:34.948055 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:34.948177 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 22:00:34.948258 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 22:00:34.948626 10778 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49222
I20260925 22:00:34.949532 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10851
I20260925 22:00:34.954249 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10991
I20260925 22:00:34.958645 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 11131
I20260925 22:00:34.963328 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 10747
2026-09-25T22:00:34Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2460 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:35 dist-test-slave-794d krb5kdc[11353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:35 dist-test-slave-794d krb5kdc[11353](info): set up 2 sockets
Sep 25 22:00:35 dist-test-slave-794d krb5kdc[11353](info): commencing operation
krb5kdc: starting...
W20260925 22:00:35.932189 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:36.052431 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44913
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:44913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:36.133594 11366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:36.133831 11366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:36.133854 11366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:36.135245 11366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:36.135313 11366 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 22:00:36.135326 11366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:36.135339 11366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:36.135350 11366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:36.135362 11366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:36.137131 11366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:44913
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:36.137370 11366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:36.137584 11366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:36.142493 11372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.142513 11374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.142513 11371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:36.142720 11366 server_base.cc:1034] running on GCE node
I20260925 22:00:36.143150 11366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:36.143453 11366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:36.144608 11366 hybrid_clock.cc:648] HybridClock initialized: now 1790373636144573 us; error 55 us; skew 500 ppm
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:36.147634 11366 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:36.148128 11366 webserver.cc:460] Webserver started at http://127.9.112.126:36577/ using document root <none> and password file <none>
I20260925 22:00:36.148432 11366 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:36.148490 11366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:36.148612 11366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:36.149752 11366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "e95e17a6ffc44d33b30527e91295cbd0"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "5dbbf1c2718877ce29419201f25d11f2"
server_key_iv: "cad3eaff22d2329bbd75d8c52af422e7"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.150092 11366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "e95e17a6ffc44d33b30527e91295cbd0"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "5dbbf1c2718877ce29419201f25d11f2"
server_key_iv: "cad3eaff22d2329bbd75d8c52af422e7"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.151425 11366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:36.152205 11381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.152419 11366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:36.152513 11366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "e95e17a6ffc44d33b30527e91295cbd0"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "5dbbf1c2718877ce29419201f25d11f2"
server_key_iv: "cad3eaff22d2329bbd75d8c52af422e7"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.152586 11366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:36.162673 11366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:36.165136 11366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:36.165301 11366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:36.170042 11366 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:44913
I20260925 22:00:36.170076 11443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:44913 every 8 connection(s)
I20260925 22:00:36.170428 11366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:36.171116 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.173507 11444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Bootstrap starting.
I20260925 22:00:36.174108 11444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.174371 11444 log.cc:824] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:36.175127 11444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: No bootstrap required, opened a new log
I20260925 22:00:36.177232 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 11366
I20260925 22:00:36.177323 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:36.177496 9665 external_mini_cluster.cc:1428] Setting key 7791dbe85ba25de4036bb82bd8773bd8
I20260925 22:00:36.177446 11444 raft_consensus.cc:348] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95e17a6ffc44d33b30527e91295cbd0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44913 } }
I20260925 22:00:36.177644 11444 raft_consensus.cc:374] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.177687 11444 raft_consensus.cc:729] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e95e17a6ffc44d33b30527e91295cbd0, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.177819 11444 consensus_queue.cc:260] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95e17a6ffc44d33b30527e91295cbd0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44913 } }
I20260925 22:00:36.177904 11444 raft_consensus.cc:388] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:36.177937 11444 raft_consensus.cc:482] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:36.177987 11444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:36.178845 11444 raft_consensus.cc:504] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95e17a6ffc44d33b30527e91295cbd0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44913 } }
I20260925 22:00:36.178962 11444 leader_election.cc:302] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e95e17a6ffc44d33b30527e91295cbd0; no voters:
I20260925 22:00:36.179128 11444 leader_election.cc:288] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:36.179160 11449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:36.179257 11449 raft_consensus.cc:686] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [term 1 LEADER]: Becoming Leader. State: Replica: e95e17a6ffc44d33b30527e91295cbd0, State: Running, Role: LEADER
I20260925 22:00:36.179405 11444 sys_catalog.cc:564] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:36.179392 11449 consensus_queue.cc:237] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95e17a6ffc44d33b30527e91295cbd0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44913 } }
I20260925 22:00:36.180035 11450 sys_catalog.cc:455] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e95e17a6ffc44d33b30527e91295cbd0. Latest consensus state: current_term: 1 leader_uuid: "e95e17a6ffc44d33b30527e91295cbd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95e17a6ffc44d33b30527e91295cbd0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44913 } } }
I20260925 22:00:36.179996 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e95e17a6ffc44d33b30527e91295cbd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95e17a6ffc44d33b30527e91295cbd0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44913 } } }
I20260925 22:00:36.180112 11450 sys_catalog.cc:458] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:36.180117 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:36.180358 11454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:36.180924 11454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:36.182650 11454 catalog_manager.cc:1347] Generated new cluster ID: af06ca70c7fc46c49af84fd9794013df
I20260925 22:00:36.182700 11454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:36.189855 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.178020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.9.112.126:44913)
0925 22:00:36.178292 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.178303 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.178339 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:00:36.178478 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.178485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.178545 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.178714 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.179526 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.181032 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.182135 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.182770 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.182897 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.185698 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.185703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.185703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.185716 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.187710 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.188175 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.188178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.188178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.188310 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.188798 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.188799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.188799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.189155 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.189327 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.189472 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.189621 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":158,"threads_started":1}
I20260925 22:00:36.193923 11454 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:36.194445 11454 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:36.205994 11454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Generated new TSK 0
I20260925 22:00:36.206251 11454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:36.236101 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44913
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:36.319571 11470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:36.319808 11470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:36.319834 11470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:36.321110 11470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:36.321159 11470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:36.321180 11470 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 22:00:36.321190 11470 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:36.321208 11470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:36.322782 11470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:44913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:36.323021 11470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:36.323252 11470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:36.328117 11478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.328125 11475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.328125 11476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:36.328461 11470 server_base.cc:1034] running on GCE node
I20260925 22:00:36.328652 11470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:36.328902 11470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:36.330070 11470 hybrid_clock.cc:648] HybridClock initialized: now 1790373636330055 us; error 32 us; skew 500 ppm
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:36.332602 11470 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:36.333081 11470 webserver.cc:460] Webserver started at http://127.9.112.65:45019/ using document root <none> and password file <none>
I20260925 22:00:36.333292 11470 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:36.333340 11470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:36.333472 11470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:36.334527 11470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "2cb0b6cb6d8d2ee65f2070c6470098c7"
server_key_iv: "e8e0cbb20a5665d5ae4384a3ed6e8501"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.334860 11470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "2cb0b6cb6d8d2ee65f2070c6470098c7"
server_key_iv: "e8e0cbb20a5665d5ae4384a3ed6e8501"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.336189 11470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:36.337061 11485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.337257 11470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:36.337363 11470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "2cb0b6cb6d8d2ee65f2070c6470098c7"
server_key_iv: "e8e0cbb20a5665d5ae4384a3ed6e8501"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.337435 11470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:36.360975 11470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:36.363343 11470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:36.363500 11470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:36.364030 11470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:36.364074 11470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.364105 11470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:36.364121 11470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.370900 11470 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:34085
I20260925 22:00:36.370996 11600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:34085 every 8 connection(s)
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:36.371368 11470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:36.374475 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.364552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:45565 (local address 127.9.112.126:44913)
0925 22:00:36.364604 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.364606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.364846 (+ 240us) server_negotiation.cc:407] Connection header received
0925 22:00:36.365346 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.365348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.365366 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.365421 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.366255 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.367048 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.368670 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.368981 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.369057 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.371836 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.371838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.371839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.371848 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.373142 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.373540 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.373542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.373542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.373599 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.373827 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.373828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.373828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.373955 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.374013 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.374289 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.374381 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:00:36.374662 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.364550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.9.112.65:45565)
0925 22:00:36.364707 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.364756 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.365246 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.365498 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.365503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.365847 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.366132 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.366139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.367180 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.367181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.368572 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.368577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.369073 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.369074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.369240 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.369943 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.369955 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.371731 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.373227 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.373234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.373237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.373472 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.373656 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.373657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.373657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.373771 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.374011 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.374018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.374166 (+ 148us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.374257 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.374434 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260925 22:00:36.375176 11601 heartbeater.cc:347] Connected to a master server at 127.9.112.126:44913
I20260925 22:00:36.375274 11601 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:36.375541 11601 heartbeater.cc:511] Master 127.9.112.126:44913 requested a full tablet report, sending...
I20260925 22:00:36.376384 11397 ts_manager.cc:194] Registered new tserver with Master: b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085)
I20260925 22:00:36.377017 11397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:45565
I20260925 22:00:36.381402 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 11470
I20260925 22:00:36.381488 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:36.381680 9665 external_mini_cluster.cc:1428] Setting key 069a9ce147a704cc750a5aec6d2ab2ed
I20260925 22:00:36.382027 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.376317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:59401 (local address 127.9.112.126:44913)
0925 22:00:36.376393 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.376396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.376509 (+ 113us) server_negotiation.cc:407] Connection header received
0925 22:00:36.376590 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.376591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.376601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.376647 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.377019 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.377572 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.378260 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.378467 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.378521 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.379445 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.379446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.379447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.379454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.380849 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.381234 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.381235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.381235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.381312 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.381509 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.381510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.381511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.381663 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.381742 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.381894 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.381963 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:00:36.382077 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.376178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.9.112.65:59401)
0925 22:00:36.376435 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.376446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.376536 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.376649 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.376650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.376886 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.376940 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.376942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.377669 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.377671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.378185 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.378186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.378526 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.378528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.378598 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.379036 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.379040 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.379371 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.380929 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.380930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.380931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.381155 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.381369 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.381369 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.381370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.381440 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.381731 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.381732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.381795 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.381861 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.381973 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:36.408660 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44913
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:36.490885 11610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:36.491127 11610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:36.491154 11610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:36.492511 11610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:36.492568 11610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:36.492599 11610 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 22:00:36.492610 11610 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:36.492626 11610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:36.494318 11610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:44913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:36.494583 11610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:36.494845 11610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:00:36.500274 11610 server_base.cc:1034] running on GCE node
W20260925 22:00:36.500243 11616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.500288 11615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.500243 11618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:36.500748 11610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:36.501042 11610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:36.502193 11610 hybrid_clock.cc:648] HybridClock initialized: now 1790373636502179 us; error 34 us; skew 500 ppm
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:36.504825 11610 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:36.505307 11610 webserver.cc:460] Webserver started at http://127.9.112.66:43405/ using document root <none> and password file <none>
I20260925 22:00:36.505537 11610 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:36.505586 11610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:36.505713 11610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:36.506764 11610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "fa08a13531e948d9b56f58928bb828f1"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "d8f8de31c00bfef977984bcbfe030526"
server_key_iv: "a67c1f3778e20ab9b523fedc3dcf0967"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.507112 11610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "fa08a13531e948d9b56f58928bb828f1"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "d8f8de31c00bfef977984bcbfe030526"
server_key_iv: "a67c1f3778e20ab9b523fedc3dcf0967"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.508539 11610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:00:36.509354 11625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.509512 11610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:36.509598 11610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "fa08a13531e948d9b56f58928bb828f1"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "d8f8de31c00bfef977984bcbfe030526"
server_key_iv: "a67c1f3778e20ab9b523fedc3dcf0967"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.509670 11610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:36.536544 11610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:36.539067 11610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:36.539225 11610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:36.539831 11610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:36.539883 11610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.539914 11610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:36.539930 11610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.546676 11610 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:45817
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:36.546746 11740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:45817 every 8 connection(s)
I20260925 22:00:36.547068 11610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:36.549988 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.540269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:42301 (local address 127.9.112.126:44913)
0925 22:00:36.540318 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.540320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.540554 (+ 234us) server_negotiation.cc:407] Connection header received
0925 22:00:36.541021 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.541022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.541034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.541112 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.542039 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.542792 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.544483 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.544781 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.544855 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.547209 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.547211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.547211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.547220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.548615 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.549070 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.549072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.549072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.549133 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.549357 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.549358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.549359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.549482 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.549536 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.549805 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.549896 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:00:36.550184 11634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.540240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.9.112.66:42301)
0925 22:00:36.540408 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.540450 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.540934 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.541205 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.541210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.541625 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.541934 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.541941 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.542919 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.542920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.544296 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.544300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.544861 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.544862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.544981 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.545577 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.545587 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.547101 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.548767 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.548775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.548777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.549005 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.549194 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.549195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.549195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.549304 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.549537 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.549543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.549687 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.549778 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.549949 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260925 22:00:36.550660 11741 heartbeater.cc:347] Connected to a master server at 127.9.112.126:44913
I20260925 22:00:36.550740 11741 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:36.550920 11741 heartbeater.cc:511] Master 127.9.112.126:44913 requested a full tablet report, sending...
I20260925 22:00:36.551321 11398 ts_manager.cc:194] Registered new tserver with Master: fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817)
I20260925 22:00:36.551785 11398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:42301
I20260925 22:00:36.554706 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 11610
I20260925 22:00:36.554791 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:36.554973 9665 external_mini_cluster.cc:1428] Setting key f2d2f41bea21d4d35db261e1d4292f0c
I20260925 22:00:36.557062 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.551708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:42091 (local address 127.9.112.126:44913)
0925 22:00:36.551750 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.551752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.551833 (+ 81us) server_negotiation.cc:407] Connection header received
0925 22:00:36.551929 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.551930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.551942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.551991 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.552327 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.552833 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.553507 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.553694 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.553748 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.554665 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.554666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.554667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.554673 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.555942 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.556247 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.556248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.556248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.556314 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.556508 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.556508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.556509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.556661 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.556719 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.556890 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.556963 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:00:36.557076 11747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.551635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.9.112.66:42091)
0925 22:00:36.551773 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.551784 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.551848 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.552004 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.552005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.552198 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.552249 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.552251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.552921 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.552922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.553438 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.553440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.553753 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.553754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.553831 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.554268 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.554272 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.554586 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.556017 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.556018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.556019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.556187 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.556401 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.556402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.556402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.556465 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.556745 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.556746 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.556799 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.556854 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.556965 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:36.581024 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44913
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:36.663192 11750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:36.663429 11750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:36.663465 11750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:36.664748 11750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:36.664801 11750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:36.664829 11750 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 22:00:36.664845 11750 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:36.664868 11750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:36.666409 11750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:44913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:36.666667 11750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:36.666891 11750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:36.672087 11755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:36.672192 11750 server_base.cc:1034] running on GCE node
W20260925 22:00:36.672087 11756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:36.672099 11758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:36.672531 11750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:36.672785 11750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:36.673925 11750 hybrid_clock.cc:648] HybridClock initialized: now 1790373636673914 us; error 22 us; skew 500 ppm
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:36.676623 11750 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:36.677101 11750 webserver.cc:460] Webserver started at http://127.9.112.67:36823/ using document root <none> and password file <none>
I20260925 22:00:36.677316 11750 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:36.677367 11750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:36.677479 11750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:36.678573 11750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "ad2bd8211239458c81313ba9129e7b94"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "aa6b4878ee5a80e0103f866f150f9fb4"
server_key_iv: "0408c4275daa8ad5e956c473c7e99051"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.678939 11750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "ad2bd8211239458c81313ba9129e7b94"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "aa6b4878ee5a80e0103f866f150f9fb4"
server_key_iv: "0408c4275daa8ad5e956c473c7e99051"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.680485 11750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:36.681520 11765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.681854 11750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:36.681941 11750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "ad2bd8211239458c81313ba9129e7b94"
format_stamp: "Formatted at 2026-09-25 22:00:36 on dist-test-slave-794d"
server_key: "aa6b4878ee5a80e0103f866f150f9fb4"
server_key_iv: "0408c4275daa8ad5e956c473c7e99051"
server_key_version: "encryptionkey@0"
I20260925 22:00:36.682024 11750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:36.690954 11750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:36.693107 11750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:36.693229 11750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:36.693722 11750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:36.693761 11750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.693787 11750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:36.693799 11750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.700065 11750 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:37615
I20260925 22:00:36.700204 11880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:37615 every 8 connection(s)
I20260925 22:00:36.700567 11750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:36.704334 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.694220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:59011 (local address 127.9.112.126:44913)
0925 22:00:36.694273 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.694275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.694587 (+ 312us) server_negotiation.cc:407] Connection header received
0925 22:00:36.695164 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.695166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.695182 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.695264 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.696086 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.696969 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.698477 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.698772 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.698851 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.701559 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.701561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.701561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.701569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.702931 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.703295 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.703296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.703296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.703353 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.703611 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.703611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.703612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.703768 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.703824 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.704183 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.704264 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:00:36.704558 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.694239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.9.112.67:59011)
0925 22:00:36.694435 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.694473 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.695041 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.695349 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.695354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.695705 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.695983 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.695990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.697071 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.697072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.698380 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.698384 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.698862 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.698863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.699022 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.699659 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.699668 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.701461 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.703003 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.703009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.703011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.703223 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.703436 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.703437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.703437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.703550 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.703872 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.703878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.704011 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.704129 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.704281 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260925 22:00:36.705180 11881 heartbeater.cc:347] Connected to a master server at 127.9.112.126:44913
I20260925 22:00:36.705274 11881 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:36.705492 11881 heartbeater.cc:511] Master 127.9.112.126:44913 requested a full tablet report, sending...
I20260925 22:00:36.705904 11397 ts_manager.cc:194] Registered new tserver with Master: ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:36.706099 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 11750
I20260925 22:00:36.706176 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:36.706342 9665 external_mini_cluster.cc:1428] Setting key 80416252c470aaca3a15ac453f25b59e
I20260925 22:00:36.706390 11397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:59011
I20260925 22:00:36.707172 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:36.713068 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.706217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.9.112.67:51877)
0925 22:00:36.706381 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.706392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.706459 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.706668 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.706668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.706934 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.706996 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.706999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.707943 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.707945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.708563 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.708566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.709118 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.709119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.709197 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.709809 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.709814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.710123 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.711922 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.711923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.711924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.712096 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.712270 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.712271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.712271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.712332 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.712723 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.712724 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.712779 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.712846 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.712965 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":70,"threads_started":1}
I20260925 22:00:36.713063 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.706284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:51877 (local address 127.9.112.126:44913)
0925 22:00:36.706325 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.706327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.706502 (+ 175us) server_negotiation.cc:407] Connection header received
0925 22:00:36.706527 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.706527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.706537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.706579 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.707071 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.707765 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.708654 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.709038 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.709123 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.710244 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.710245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.710246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.710253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.711823 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.712159 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.712161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.712161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.712214 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.712518 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.712519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.712520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.712662 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.712723 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.712879 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.712961 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:36.730491 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.723627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.9.112.126:44913)
0925 22:00:36.723690 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.723692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.723713 (+ 21us) server_negotiation.cc:407] Connection header received
0925 22:00:36.723739 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.723740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.723748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.723790 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.724334 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.724841 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.725580 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.725765 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.725820 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.728227 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.728229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.728230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.728237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.729516 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.729890 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.729891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.729892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.729943 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.730084 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.730084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.730084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.730218 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.730288 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.730365 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.730424 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:00:36.732160 11397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:36.734211 11397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:36.741074 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.736787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.0.0.1:33550)
0925 22:00:36.737078 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.737089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.737142 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.737420 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.737423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.737426 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:36.737510 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.737512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.738590 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.738591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.739826 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.739829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.740738 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.740739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.740833 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.740849 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.740924 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.740985 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260925 22:00:36.741115 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.736910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.9.112.66:45817)
0925 22:00:36.737218 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.737224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.737234 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:36.737274 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.737276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.737294 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.737376 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:36.737716 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.738509 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.739947 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.740662 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.740737 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.740808 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.740985 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.741024 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":78,"threads_started":1}
I20260925 22:00:36.741367 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.736771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34085 (local address 127.0.0.1:54620)
0925 22:00:36.736956 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.737000 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.737070 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.737703 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.737703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.737706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:36.737751 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.737753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.739095 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.739096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.740074 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.740076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.741084 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.741085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.741194 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.741199 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.741232 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.741279 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":50,"threads_started":1}
I20260925 22:00:36.741703 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.737190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55400 (local address 127.9.112.67:37615)
0925 22:00:36.737337 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.737340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.737350 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:36.737422 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.737424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.737439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.737519 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:36.737832 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.738974 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.740263 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.740991 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.741054 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.741122 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.741563 (+ 441us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.741604 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260925 22:00:36.741714 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.736874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54620 (local address 127.9.112.65:34085)
0925 22:00:36.737505 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.737509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.737518 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:36.737550 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.737552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.737567 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.737643 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:36.737812 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.738996 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.740161 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.740985 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.741451 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.741516 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.741554 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.741595 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":77,"threads_started":1}
I20260925 22:00:36.741367 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.736996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:37615 (local address 127.0.0.1:55400)
0925 22:00:36.737095 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.737104 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.737342 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.737525 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.737526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.737528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:36.737585 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.737587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.739071 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.739072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.740184 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.740186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.741151 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.741152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.741213 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.741216 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.741258 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.741292 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":47,"thread_start_us":63,"threads_started":1}
I20260925 22:00:36.741997 11675 tablet_service.cc:1467] Processing CreateTablet for tablet aa4372a2afe24251a651efceb55f7404 (DEFAULT_TABLE table=test-table [id=9e11c3470fb44aa8b8eac7bbee96c244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:36.742596 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa4372a2afe24251a651efceb55f7404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.742486 11535 tablet_service.cc:1467] Processing CreateTablet for tablet aa4372a2afe24251a651efceb55f7404 (DEFAULT_TABLE table=test-table [id=9e11c3470fb44aa8b8eac7bbee96c244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:36.743145 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa4372a2afe24251a651efceb55f7404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.745775 11904 tablet_bootstrap.cc:492] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Bootstrap starting.
I20260925 22:00:36.746506 11904 tablet_bootstrap.cc:654] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.746403 11815 tablet_service.cc:1467] Processing CreateTablet for tablet aa4372a2afe24251a651efceb55f7404 (DEFAULT_TABLE table=test-table [id=9e11c3470fb44aa8b8eac7bbee96c244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:36.746760 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa4372a2afe24251a651efceb55f7404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.746841 11904 log.cc:824] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:36.747640 11904 tablet_bootstrap.cc:492] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: No bootstrap required, opened a new log
I20260925 22:00:36.747716 11904 ts_tablet_manager.cc:1397] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:36.748566 11905 tablet_bootstrap.cc:492] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Bootstrap starting.
I20260925 22:00:36.749147 11905 tablet_bootstrap.cc:654] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.749449 11905 log.cc:824] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:36.749734 11904 raft_consensus.cc:348] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.749913 11904 raft_consensus.cc:374] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.749982 11904 raft_consensus.cc:729] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.750144 11905 tablet_bootstrap.cc:492] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: No bootstrap required, opened a new log
I20260925 22:00:36.750152 11904 consensus_queue.cc:260] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.750262 11908 tablet_bootstrap.cc:492] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Bootstrap starting.
I20260925 22:00:36.750278 11905 ts_tablet_manager.cc:1397] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:36.750448 11904 ts_tablet_manager.cc:1428] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:00:36.750578 11741 heartbeater.cc:503] Master 127.9.112.126:44913 was elected leader, sending a full tablet report...
I20260925 22:00:36.751174 11908 tablet_bootstrap.cc:654] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.751443 11908 log.cc:824] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:36.752177 11908 tablet_bootstrap.cc:492] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: No bootstrap required, opened a new log
I20260925 22:00:36.752231 11908 ts_tablet_manager.cc:1397] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:36.752188 11905 raft_consensus.cc:348] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.752409 11905 raft_consensus.cc:374] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.752455 11905 raft_consensus.cc:729] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c1f29642b54a7d870ad1a1f3800c44, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.752571 11905 consensus_queue.cc:260] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.752822 11905 ts_tablet_manager.cc:1428] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:36.752890 11601 heartbeater.cc:503] Master 127.9.112.126:44913 was elected leader, sending a full tablet report...
I20260925 22:00:36.753919 11908 raft_consensus.cc:348] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.754065 11908 raft_consensus.cc:374] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.754094 11908 raft_consensus.cc:729] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.754208 11908 consensus_queue.cc:260] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.754438 11908 ts_tablet_manager.cc:1428] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:36.754488 11881 heartbeater.cc:503] Master 127.9.112.126:44913 was elected leader, sending a full tablet report...
W20260925 22:00:36.797657 11742 tablet.cc:2369] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:36.817109 11912 raft_consensus.cc:482] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:36.817236 11912 raft_consensus.cc:504] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.817533 11912 leader_election.cc:288] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817)
I20260925 22:00:36.821539 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.817833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:53237 (local address 127.9.112.66:45817)
0925 22:00:36.817872 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.817874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.817883 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:36.817960 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.817961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.817971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.818032 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:36.818196 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.818938 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.820539 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.821123 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.821207 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.821216 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.821420 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.821460 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:00:36.821539 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.817887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:48809 (local address 127.9.112.65:34085)
0925 22:00:36.817929 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.817931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.818079 (+ 148us) server_negotiation.cc:407] Connection header received
0925 22:00:36.818106 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.818107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.818114 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.818154 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:36.818280 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.818962 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.820540 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.821135 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.821207 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.821216 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.821425 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.821462 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:36.821576 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.817736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.9.112.67:53237)
0925 22:00:36.817799 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.817812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.817881 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.818049 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.818050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.818052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:36.818122 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.818124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.819040 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.819041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.820420 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.820423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.821235 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.821237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.821327 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.821331 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.821415 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.821470 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":66}
I20260925 22:00:36.821581 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.817745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34085 (local address 127.9.112.67:48809)
0925 22:00:36.818005 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.818014 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.818067 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.818171 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.818172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.818173 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:36.818225 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.818226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.819040 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.819041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.820420 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.820423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.821258 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.821259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.821349 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.821352 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.821415 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.821470 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":180,"thread_start_us":115,"threads_started":1}
I20260925 22:00:36.821970 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4372a2afe24251a651efceb55f7404" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1" is_pre_election: true
I20260925 22:00:36.822160 11695 raft_consensus.cc:2454] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 0.
I20260925 22:00:36.822129 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4372a2afe24251a651efceb55f7404" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" is_pre_election: true
I20260925 22:00:36.822273 11555 raft_consensus.cc:2454] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 0.
I20260925 22:00:36.822408 11769 leader_election.cc:302] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, fa08a13531e948d9b56f58928bb828f1; no voters:
I20260925 22:00:36.822566 11912 raft_consensus.cc:2781] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:36.822623 11912 raft_consensus.cc:482] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:36.822638 11912 raft_consensus.cc:3037] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:36.823345 11912 raft_consensus.cc:504] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.823486 11912 leader_election.cc:288] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 election: Requested vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817)
I20260925 22:00:36.823655 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4372a2afe24251a651efceb55f7404" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
I20260925 22:00:36.823741 11555 raft_consensus.cc:3037] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:36.824476 11555 raft_consensus.cc:2454] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 1.
I20260925 22:00:36.824576 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa4372a2afe24251a651efceb55f7404" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1"
I20260925 22:00:36.824676 11695 raft_consensus.cc:3037] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:36.824687 11766 leader_election.cc:302] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, b6c1f29642b54a7d870ad1a1f3800c44; no voters:
I20260925 22:00:36.825343 11695 raft_consensus.cc:2454] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 1.
I20260925 22:00:36.825441 11912 raft_consensus.cc:2781] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:36.825603 11912 raft_consensus.cc:686] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 1 LEADER]: Becoming Leader. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Running, Role: LEADER
I20260925 22:00:36.825703 11912 consensus_queue.cc:237] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.826588 11397 catalog_manager.cc:5495] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2bd8211239458c81313ba9129e7b94 (127.9.112.67). New cstate: current_term: 1 leader_uuid: "ad2bd8211239458c81313ba9129e7b94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } health_report { overall_health: HEALTHY } } }
Sep 25 22:00:36 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:36.869092 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.861216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44913 (local address 127.0.0.1:51140)
0925 22:00:36.861410 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:36.861420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:36.861491 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:36.861623 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:36.861624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:36.861988 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:36.862051 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.862054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.862725 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.862726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.863246 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:36.863247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.863627 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.863627 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:36.863694 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.864250 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:36.864254 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:36.865716 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:36.867899 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.867909 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.867912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.868144 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.868436 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:36.868437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:36.868437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:36.868515 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:36.868802 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:36.868809 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:36.868858 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:00:36.868908 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:00:36.869013 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260925 22:00:36.869175 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:36.861284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.9.112.126:44913)
0925 22:00:36.861359 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:36.861361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:36.861480 (+ 119us) server_negotiation.cc:407] Connection header received
0925 22:00:36.861558 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:36.861558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:36.861568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:36.861607 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:36.862138 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.862640 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.863320 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:36.863509 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:36.863624 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:36.865833 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:36.865834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:36.865834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:36.865842 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:36.867759 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.868246 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.868248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.868248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.868332 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:36.868604 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:36.868605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:36.868606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:36.868736 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:36.868801 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:36.868966 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 22:00:36.869068 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":46}
I20260925 22:00:36.871353 11397 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:51140:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 22:00:36.871588 11397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 22:00:36.872076 11602 tablet.cc:2369] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:36.873198 11535 tablet_service.cc:1467] Processing CreateTablet for tablet 8669f561bfa74955ae0ecd2e7f6b434d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d960462b1134068a6b6f828b771f9ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:00:36.873363 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8669f561bfa74955ae0ecd2e7f6b434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.873377 11815 tablet_service.cc:1467] Processing CreateTablet for tablet 8669f561bfa74955ae0ecd2e7f6b434d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d960462b1134068a6b6f828b771f9ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:00:36.873548 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8669f561bfa74955ae0ecd2e7f6b434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.874648 11905 tablet_bootstrap.cc:492] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: Bootstrap starting.
I20260925 22:00:36.874774 11908 tablet_bootstrap.cc:492] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: Bootstrap starting.
I20260925 22:00:36.874789 11675 tablet_service.cc:1467] Processing CreateTablet for tablet 8669f561bfa74955ae0ecd2e7f6b434d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d960462b1134068a6b6f828b771f9ff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:00:36.874946 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8669f561bfa74955ae0ecd2e7f6b434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:36.875200 11908 tablet_bootstrap.cc:654] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.875329 11905 tablet_bootstrap.cc:654] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.875880 11908 tablet_bootstrap.cc:492] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: No bootstrap required, opened a new log
I20260925 22:00:36.875936 11908 ts_tablet_manager.cc:1397] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:36.876073 11908 raft_consensus.cc:348] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.876163 11908 raft_consensus.cc:374] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.876185 11908 raft_consensus.cc:729] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.876236 11908 consensus_queue.cc:260] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.876380 11908 ts_tablet_manager.cc:1428] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:36.876523 11904 tablet_bootstrap.cc:492] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: Bootstrap starting.
I20260925 22:00:36.876526 11912 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:36.876590 11912 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:36.876792 11905 tablet_bootstrap.cc:492] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: No bootstrap required, opened a new log
I20260925 22:00:36.876837 11905 ts_tablet_manager.cc:1397] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:36.876959 11905 raft_consensus.cc:348] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.877017 11905 raft_consensus.cc:374] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.877034 11905 raft_consensus.cc:729] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c1f29642b54a7d870ad1a1f3800c44, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.877082 11905 consensus_queue.cc:260] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.877164 11905 ts_tablet_manager.cc:1428] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.877200 11904 tablet_bootstrap.cc:654] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:36.877259 11911 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:36.877375 11911 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:36.877969 11904 tablet_bootstrap.cc:492] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: No bootstrap required, opened a new log
I20260925 22:00:36.878017 11904 ts_tablet_manager.cc:1397] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:36.878137 11904 raft_consensus.cc:348] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.878182 11904 raft_consensus.cc:374] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:36.878196 11904 raft_consensus.cc:729] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Initialized, Role: FOLLOWER
I20260925 22:00:36.878230 11904 consensus_queue.cc:260] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:36.878295 11904 ts_tablet_manager.cc:1428] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:36.878383 11909 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:36.878485 11909 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 22:00:36.951256 11882 tablet.cc:2369] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:37.058074 11909 raft_consensus.cc:482] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:37.058184 11909 raft_consensus.cc:504] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.058555 11909 leader_election.cc:288] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:37.059970 11911 raft_consensus.cc:482] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:37.060050 11911 raft_consensus.cc:504] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.060405 11911 leader_election.cc:288] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:37.061838 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.058861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:47397 (local address 127.9.112.67:37615)
0925 22:00:37.058911 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.058913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.059002 (+ 89us) server_negotiation.cc:407] Connection header received
0925 22:00:37.059072 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.059073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.059081 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.059124 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:37.059298 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.059763 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061063 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.061501 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061542 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.061550 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.061731 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.061772 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:00:37.061846 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.058787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:37615 (local address 127.9.112.66:47397)
0925 22:00:37.058917 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.058928 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.058998 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.059152 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.059152 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.059163 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:37.059229 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.059231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.059837 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.059838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.060952 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.060955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.061602 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.061657 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.061661 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.061710 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.061754 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260925 22:00:37.061982 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.058969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:46695 (local address 127.9.112.65:34085)
0925 22:00:37.059013 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.059014 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.059038 (+ 24us) server_negotiation.cc:407] Connection header received
0925 22:00:37.059069 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.059070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.059079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.059146 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:37.059284 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.059774 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061065 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.061634 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061678 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.061686 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.061883 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.061919 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:00:37.062191 11835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8669f561bfa74955ae0ecd2e7f6b434d" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94" is_pre_election: true
I20260925 22:00:37.062322 11835 raft_consensus.cc:2454] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 0.
I20260925 22:00:37.062543 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.058795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34085 (local address 127.9.112.66:46695)
0925 22:00:37.058937 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.058947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.059002 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.059148 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.059148 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.059151 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:37.059223 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.059225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.059859 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.059860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.060978 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.060980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.061741 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.061808 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.061812 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.062060 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.062458 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":50,"thread_start_us":66,"threads_started":1}
I20260925 22:00:37.062515 11629 leader_election.cc:302] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, fa08a13531e948d9b56f58928bb828f1; no voters:
I20260925 22:00:37.062788 11909 raft_consensus.cc:2781] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:37.062820 11909 raft_consensus.cc:482] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:37.062837 11909 raft_consensus.cc:3037] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:37.062884 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8669f561bfa74955ae0ecd2e7f6b434d" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" is_pre_election: true
I20260925 22:00:37.062978 11555 raft_consensus.cc:2454] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 0.
I20260925 22:00:37.063719 11909 raft_consensus.cc:504] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.063800 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.060542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.9.112.65:34989)
0925 22:00:37.060656 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.060667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.060727 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.060981 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.060982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.060984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:37.061045 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.061047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.061863 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.061863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.062906 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.062909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.063565 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.063566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.063628 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.063632 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.063683 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.063735 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260925 22:00:37.063812 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.060840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:34989 (local address 127.9.112.66:45817)
0925 22:00:37.060877 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.060879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.060885 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:00:37.060902 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.060903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.060910 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.060948 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:37.061115 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.061781 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.062993 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.063456 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.063502 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.063509 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.063710 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.063750 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:00:37.063849 11909 leader_election.cc:288] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 election: Requested vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:37.064052 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8669f561bfa74955ae0ecd2e7f6b434d" candidate_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1" is_pre_election: true
I20260925 22:00:37.064150 11695 raft_consensus.cc:2379] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b6c1f29642b54a7d870ad1a1f3800c44 in current term 1: Already voted for candidate fa08a13531e948d9b56f58928bb828f1 in this term.
I20260925 22:00:37.064184 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8669f561bfa74955ae0ecd2e7f6b434d" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
I20260925 22:00:37.064248 11555 raft_consensus.cc:3037] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:37.064399 11835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8669f561bfa74955ae0ecd2e7f6b434d" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94"
I20260925 22:00:37.064471 11835 raft_consensus.cc:3037] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:37.065007 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.060901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:47975 (local address 127.9.112.67:37615)
0925 22:00:37.061921 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.061923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.061931 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:37.061949 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.061950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.061959 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.062008 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:37.062473 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.063053 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.064131 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.064643 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.064724 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.064731 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.064907 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.064943 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1005,"thread_start_us":82,"threads_started":1}
I20260925 22:00:37.065024 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.060696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:37615 (local address 127.9.112.65:47975)
0925 22:00:37.061035 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.061045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.061090 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.062316 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.062317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.062320 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:37.062377 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.062379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.063170 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.063171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.064023 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.064026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.064751 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.064752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.064823 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.064828 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.064898 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.064937 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":59,"threads_started":1}
I20260925 22:00:37.065038 11555 raft_consensus.cc:2454] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 1.
I20260925 22:00:37.065233 11835 raft_consensus.cc:2454] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 1.
I20260925 22:00:37.065335 11626 leader_election.cc:302] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6c1f29642b54a7d870ad1a1f3800c44, fa08a13531e948d9b56f58928bb828f1; no voters:
I20260925 22:00:37.065357 11834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8669f561bfa74955ae0ecd2e7f6b434d" candidate_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94" is_pre_election: true
I20260925 22:00:37.065434 11909 raft_consensus.cc:2781] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:37.065479 11834 raft_consensus.cc:2379] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b6c1f29642b54a7d870ad1a1f3800c44 in current term 1: Already voted for candidate fa08a13531e948d9b56f58928bb828f1 in this term.
I20260925 22:00:37.065551 11909 raft_consensus.cc:686] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [term 1 LEADER]: Becoming Leader. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Running, Role: LEADER
I20260925 22:00:37.065644 11909 consensus_queue.cc:237] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.065802 11489 leader_election.cc:302] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6c1f29642b54a7d870ad1a1f3800c44; no voters: ad2bd8211239458c81313ba9129e7b94, fa08a13531e948d9b56f58928bb828f1
I20260925 22:00:37.065948 11911 raft_consensus.cc:2726] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 22:00:37.066062 11930 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: TxnStatusTablet state changed. Reason: New leader fa08a13531e948d9b56f58928bb828f1. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.066143 11930 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:00:37.066376 11932 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 22:00:37.066480 11932 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 22:00:37.066448 11397 catalog_manager.cc:5495] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa08a13531e948d9b56f58928bb828f1 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:37.106474 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.103839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.0.0.1:33560)
0925 22:00:37.103954 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.103966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.104031 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.104207 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.104208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.104210 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:37.104262 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.104265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.104970 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.104971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.105604 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.105607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.105928 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.105929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.105995 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.106012 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.106293 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.106304 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.106354 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.106403 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 22:00:37.106472 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.103905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.9.112.66:45817)
0925 22:00:37.103948 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.103950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.104017 (+ 67us) server_negotiation.cc:407] Connection header received
0925 22:00:37.104112 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.104113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.104135 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.104195 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:37.104318 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.104890 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.105678 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.105873 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.105923 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.106107 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.106242 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.106312 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.106365 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.106386 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:00:37.108302 11834 raft_consensus.cc:1261] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Refusing update from remote peer fa08a13531e948d9b56f58928bb828f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:37.108711 11555 raft_consensus.cc:1261] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Refusing update from remote peer fa08a13531e948d9b56f58928bb828f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:37.108997 11930 consensus_queue.cc:1035] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:37.109138 11909 consensus_queue.cc:1035] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:37.110452 11912 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: TxnStatusTablet state changed. Reason: New leader fa08a13531e948d9b56f58928bb828f1. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.110543 11912 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:37.110492 11911 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: TxnStatusTablet state changed. Reason: New leader fa08a13531e948d9b56f58928bb828f1. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.110587 11911 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:37.111857 11909 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.111948 11909 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:00:37.112002 11909 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.112040 11909 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P fa08a13531e948d9b56f58928bb828f1: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:00:37.112238 11912 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.112329 11912 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P ad2bd8211239458c81313ba9129e7b94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:37.112509 11937 mvcc.cc:205] Tried to move back new op lower bound from 7333370417593315328 to 7333370417421500416. Current Snapshot: MvccSnapshot[applied={T|T < 7333370417593315328}]
I20260925 22:00:37.112622 11911 tablet_replica.cc:439] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } } }
I20260925 22:00:37.112689 11911 tablet_replica.cc:442] T 8669f561bfa74955ae0ecd2e7f6b434d P b6c1f29642b54a7d870ad1a1f3800c44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:00:37.112692 11935 mvcc.cc:205] Tried to move back new op lower bound from 7333370417593315328 to 7333370417421500416. Current Snapshot: MvccSnapshot[applied={T|T < 7333370417593315328}]
I20260925 22:00:37.113765 11936 mvcc.cc:205] Tried to move back new op lower bound from 7333370417593315328 to 7333370417421500416. Current Snapshot: MvccSnapshot[applied={T|T < 7333370417593315328}]
I20260925 22:00:37.117437 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.114595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.9.112.67:37615)
0925 22:00:37.114657 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.114659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.114668 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:37.114697 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.114698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.114715 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.114771 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:37.114918 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.115386 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.116587 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.116930 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.116989 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.117164 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.117262 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.117333 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.117354 (+ 21us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.117378 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:00:37.123322 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.120254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.9.112.67:41037)
0925 22:00:37.120333 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.120376 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.120414 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.120531 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.120532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.120533 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:37.120610 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.120612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.121339 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.121340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.122312 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.122315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.122770 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.122771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.122845 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.122867 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.123126 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.123129 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.123174 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.123227 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 22:00:37.123620 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.120378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:41037 (local address 127.9.112.66:45817)
0925 22:00:37.120434 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.120436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.120446 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:37.120471 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.120471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.120480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.120528 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:37.120719 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.121229 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.122441 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.122699 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.122760 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.123014 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.123055 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.123106 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.123526 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.123556 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:00:37.126308 11695 raft_consensus.cc:1261] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Refusing update from remote peer ad2bd8211239458c81313ba9129e7b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:37.126304 11555 raft_consensus.cc:1261] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Refusing update from remote peer ad2bd8211239458c81313ba9129e7b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:37.126587 11914 consensus_queue.cc:1035] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:37.126722 11912 consensus_queue.cc:1035] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:37.136330 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.133053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:58707 (local address 127.9.112.67:37615)
0925 22:00:37.133093 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:37.133095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:37.133235 (+ 140us) server_negotiation.cc:407] Connection header received
0925 22:00:37.133665 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:37.133666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:37.133677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:37.133728 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:37.133931 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.134503 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.135635 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.135881 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.135943 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.136121 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.136149 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.136182 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:37.136244 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 22:00:37.136260 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:00:37.136408 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:37.133019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:37615 (local address 127.9.112.66:58707)
0925 22:00:37.133172 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:37.133183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:37.133309 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:37.133737 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:37.133738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:37.133740 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:37.133797 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.133799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.134589 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.134590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.135521 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:37.135524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:37.135961 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:37.135962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:37.136024 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:37.136040 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:37.136186 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:37.136187 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:00:37.136241 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:00:37.136290 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260925 22:00:37.145829 11398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:37.145963 11398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:37.146780 11398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Sending DeleteTablet for 3 replicas of tablet aa4372a2afe24251a651efceb55f7404
I20260925 22:00:37.147115 11535 tablet_service.cc:1514] Processing DeleteTablet for tablet aa4372a2afe24251a651efceb55f7404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:37 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:54620
I20260925 22:00:37.147148 11675 tablet_service.cc:1514] Processing DeleteTablet for tablet aa4372a2afe24251a651efceb55f7404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:37 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:33550
I20260925 22:00:37.147393 11955 tablet_replica.cc:330] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: stopping tablet replica
I20260925 22:00:37.147449 11815 tablet_service.cc:1514] Processing DeleteTablet for tablet aa4372a2afe24251a651efceb55f7404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:37 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:55400
I20260925 22:00:37.147505 11955 raft_consensus.cc:2229] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:37.147567 11955 raft_consensus.cc:2258] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:37.148051 11955 ts_tablet_manager.cc:1905] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:37.148072 11398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:37.148232 11398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:37.149556 11954 tablet_replica.cc:330] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: stopping tablet replica
I20260925 22:00:37.149747 11956 tablet_replica.cc:330] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: stopping tablet replica
I20260925 22:00:37.149914 11956 raft_consensus.cc:2229] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:37.149996 11535 tablet_service.cc:1467] Processing CreateTablet for tablet cec19dff6bca47f5b950a928b6a00a2b (DEFAULT_TABLE table=test-table [id=6f0de7f2d4d04e488d8c9ef0431d542e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:37.150056 11954 raft_consensus.cc:2229] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:37.150121 11954 raft_consensus.cc:2258] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:37.150138 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec19dff6bca47f5b950a928b6a00a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:37.150018 11815 tablet_service.cc:1467] Processing CreateTablet for tablet cec19dff6bca47f5b950a928b6a00a2b (DEFAULT_TABLE table=test-table [id=6f0de7f2d4d04e488d8c9ef0431d542e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:37.150283 11956 raft_consensus.cc:2258] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:37.150332 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec19dff6bca47f5b950a928b6a00a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:37.150748 11956 ts_tablet_manager.cc:1905] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:37.151317 11675 tablet_service.cc:1467] Processing CreateTablet for tablet cec19dff6bca47f5b950a928b6a00a2b (DEFAULT_TABLE table=test-table [id=6f0de7f2d4d04e488d8c9ef0431d542e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:37.151422 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec19dff6bca47f5b950a928b6a00a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:37.151620 11908 tablet_bootstrap.cc:492] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Bootstrap starting.
I20260925 22:00:37.152025 11908 tablet_bootstrap.cc:654] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:37.150141 11955 ts_tablet_manager.cc:1918] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:37.152506 11955 log.cc:1196] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/aa4372a2afe24251a651efceb55f7404
I20260925 22:00:37.152771 11904 tablet_bootstrap.cc:492] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Bootstrap starting.
I20260925 22:00:37.152792 11905 tablet_bootstrap.cc:492] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Bootstrap starting.
I20260925 22:00:37.153169 11905 tablet_bootstrap.cc:654] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:37.153296 11904 tablet_bootstrap.cc:654] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:37.152791 11955 ts_tablet_manager.cc:1939] T aa4372a2afe24251a651efceb55f7404 P b6c1f29642b54a7d870ad1a1f3800c44: Deleting consensus metadata
I20260925 22:00:37.153852 11954 ts_tablet_manager.cc:1905] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:37.154029 11956 ts_tablet_manager.cc:1918] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:37.154177 11956 log.cc:1196] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/aa4372a2afe24251a651efceb55f7404
I20260925 22:00:37.154392 11382 catalog_manager.cc:4841] TS b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085): tablet aa4372a2afe24251a651efceb55f7404 (table test-table [id=9e11c3470fb44aa8b8eac7bbee96c244]) successfully deleted
I20260925 22:00:37.154525 11956 ts_tablet_manager.cc:1939] T aa4372a2afe24251a651efceb55f7404 P ad2bd8211239458c81313ba9129e7b94: Deleting consensus metadata
I20260925 22:00:37.154979 11385 catalog_manager.cc:4841] TS ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615): tablet aa4372a2afe24251a651efceb55f7404 (table test-table [id=9e11c3470fb44aa8b8eac7bbee96c244]) successfully deleted
I20260925 22:00:37.155555 11908 tablet_bootstrap.cc:492] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: No bootstrap required, opened a new log
I20260925 22:00:37.155601 11908 ts_tablet_manager.cc:1397] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 22:00:37.155740 11908 raft_consensus.cc:348] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.155825 11908 raft_consensus.cc:374] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:37.155845 11908 raft_consensus.cc:729] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Initialized, Role: FOLLOWER
I20260925 22:00:37.155900 11908 consensus_queue.cc:260] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.155982 11908 ts_tablet_manager.cc:1428] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:37.156857 11954 ts_tablet_manager.cc:1918] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:37.156985 11954 log.cc:1196] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/aa4372a2afe24251a651efceb55f7404
I20260925 22:00:37.157307 11904 tablet_bootstrap.cc:492] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: No bootstrap required, opened a new log
I20260925 22:00:37.157348 11954 ts_tablet_manager.cc:1939] T aa4372a2afe24251a651efceb55f7404 P fa08a13531e948d9b56f58928bb828f1: Deleting consensus metadata
I20260925 22:00:37.157356 11904 ts_tablet_manager.cc:1397] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 22:00:37.157510 11904 raft_consensus.cc:348] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.157574 11904 raft_consensus.cc:374] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:37.157593 11904 raft_consensus.cc:729] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Initialized, Role: FOLLOWER
I20260925 22:00:37.157631 11904 consensus_queue.cc:260] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.157716 11904 ts_tablet_manager.cc:1428] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:37.157750 11385 catalog_manager.cc:4841] TS fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817): tablet aa4372a2afe24251a651efceb55f7404 (table test-table [id=9e11c3470fb44aa8b8eac7bbee96c244]) successfully deleted
I20260925 22:00:37.158150 11905 tablet_bootstrap.cc:492] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: No bootstrap required, opened a new log
I20260925 22:00:37.158218 11905 ts_tablet_manager.cc:1397] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 22:00:37.158371 11905 raft_consensus.cc:348] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.158450 11905 raft_consensus.cc:374] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:37.158471 11905 raft_consensus.cc:729] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c1f29642b54a7d870ad1a1f3800c44, State: Initialized, Role: FOLLOWER
I20260925 22:00:37.158514 11905 consensus_queue.cc:260] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.158599 11905 ts_tablet_manager.cc:1428] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:37.191642 11951 raft_consensus.cc:482] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:37.191762 11951 raft_consensus.cc:504] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.191956 11951 leader_election.cc:288] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817)
I20260925 22:00:37.192186 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec19dff6bca47f5b950a928b6a00a2b" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" is_pre_election: true
I20260925 22:00:37.192306 11555 raft_consensus.cc:2454] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 0.
I20260925 22:00:37.192265 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec19dff6bca47f5b950a928b6a00a2b" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1" is_pre_election: true
I20260925 22:00:37.192387 11695 raft_consensus.cc:2454] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 0.
I20260925 22:00:37.192484 11766 leader_election.cc:302] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, b6c1f29642b54a7d870ad1a1f3800c44; no voters:
I20260925 22:00:37.192622 11951 raft_consensus.cc:2781] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:37.192655 11951 raft_consensus.cc:482] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:37.192674 11951 raft_consensus.cc:3037] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:37.193543 11951 raft_consensus.cc:504] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.193693 11951 leader_election.cc:288] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 election: Requested vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817)
I20260925 22:00:37.193892 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec19dff6bca47f5b950a928b6a00a2b" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
I20260925 22:00:37.193918 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec19dff6bca47f5b950a928b6a00a2b" candidate_uuid: "ad2bd8211239458c81313ba9129e7b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1"
I20260925 22:00:37.193965 11555 raft_consensus.cc:3037] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:37.193993 11695 raft_consensus.cc:3037] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:37.194731 11555 raft_consensus.cc:2454] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 1.
I20260925 22:00:37.194828 11695 raft_consensus.cc:2454] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2bd8211239458c81313ba9129e7b94 in term 1.
I20260925 22:00:37.194900 11766 leader_election.cc:302] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, b6c1f29642b54a7d870ad1a1f3800c44; no voters:
I20260925 22:00:37.194973 11951 raft_consensus.cc:2781] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:37.195013 11951 raft_consensus.cc:686] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 1 LEADER]: Becoming Leader. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Running, Role: LEADER
I20260925 22:00:37.195057 11951 consensus_queue.cc:237] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:37.195739 11398 catalog_manager.cc:5495] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2bd8211239458c81313ba9129e7b94 (127.9.112.67). New cstate: current_term: 1 leader_uuid: "ad2bd8211239458c81313ba9129e7b94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } health_report { overall_health: HEALTHY } } }
I20260925 22:00:37.202732 11555 raft_consensus.cc:1261] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Refusing update from remote peer ad2bd8211239458c81313ba9129e7b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:37.202965 11951 consensus_queue.cc:1035] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:37.203558 11695 raft_consensus.cc:1261] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Refusing update from remote peer ad2bd8211239458c81313ba9129e7b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:37.203773 11951 consensus_queue.cc:1035] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 22:00:37.210692 11936 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 22:00:38.734524 11398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:38.734674 11398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:38.735841 11398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Sending DeleteTablet for 3 replicas of tablet cec19dff6bca47f5b950a928b6a00a2b
I20260925 22:00:38.736155 11535 tablet_service.cc:1514] Processing DeleteTablet for tablet cec19dff6bca47f5b950a928b6a00a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:38 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:54620
I20260925 22:00:38.736197 11675 tablet_service.cc:1514] Processing DeleteTablet for tablet cec19dff6bca47f5b950a928b6a00a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:38 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:33550
I20260925 22:00:38.736409 11815 tablet_service.cc:1514] Processing DeleteTablet for tablet cec19dff6bca47f5b950a928b6a00a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:38 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:55400
I20260925 22:00:38.736514 11987 tablet_replica.cc:330] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: stopping tablet replica
I20260925 22:00:38.736524 11986 tablet_replica.cc:330] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: stopping tablet replica
I20260925 22:00:38.736591 11987 raft_consensus.cc:2229] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:38.736591 11986 raft_consensus.cc:2229] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:38.736645 11987 raft_consensus.cc:2258] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:38.736645 11986 raft_consensus.cc:2258] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:38.736830 11988 tablet_replica.cc:330] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: stopping tablet replica
I20260925 22:00:38.736922 11988 raft_consensus.cc:2229] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:38.737004 11988 raft_consensus.cc:2258] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:38.737057 11986 ts_tablet_manager.cc:1905] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:38.737360 11988 ts_tablet_manager.cc:1905] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:38.738998 11986 ts_tablet_manager.cc:1918] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:00:38.738997 11988 ts_tablet_manager.cc:1918] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:00:38.739055 11988 log.cc:1196] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/cec19dff6bca47f5b950a928b6a00a2b
I20260925 22:00:38.739219 11986 log.cc:1196] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/cec19dff6bca47f5b950a928b6a00a2b
I20260925 22:00:38.739279 11988 ts_tablet_manager.cc:1939] T cec19dff6bca47f5b950a928b6a00a2b P ad2bd8211239458c81313ba9129e7b94: Deleting consensus metadata
I20260925 22:00:38.739475 11986 ts_tablet_manager.cc:1939] T cec19dff6bca47f5b950a928b6a00a2b P b6c1f29642b54a7d870ad1a1f3800c44: Deleting consensus metadata
I20260925 22:00:38.739908 11385 catalog_manager.cc:4841] TS ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615): tablet cec19dff6bca47f5b950a928b6a00a2b (table test-table [id=6f0de7f2d4d04e488d8c9ef0431d542e]) successfully deleted
I20260925 22:00:38.740084 11382 catalog_manager.cc:4841] TS b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085): tablet cec19dff6bca47f5b950a928b6a00a2b (table test-table [id=6f0de7f2d4d04e488d8c9ef0431d542e]) successfully deleted
I20260925 22:00:38.740607 11987 ts_tablet_manager.cc:1905] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:38.742456 11987 ts_tablet_manager.cc:1918] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:00:38.742533 11987 log.cc:1196] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/cec19dff6bca47f5b950a928b6a00a2b
I20260925 22:00:38.742807 11987 ts_tablet_manager.cc:1939] T cec19dff6bca47f5b950a928b6a00a2b P fa08a13531e948d9b56f58928bb828f1: Deleting consensus metadata
I20260925 22:00:38.744014 11385 catalog_manager.cc:4841] TS fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817): tablet cec19dff6bca47f5b950a928b6a00a2b (table test-table [id=6f0de7f2d4d04e488d8c9ef0431d542e]) successfully deleted
Sep 25 22:00:38 dist-test-slave-794d krb5kdc[11353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:00:38 dist-test-slave-794d krb5kdc[11353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:38.766228 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:38.757191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.9.112.126:44913)
0925 22:00:38.757389 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:38.757391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:38.757399 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:38.757427 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:38.757427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:38.757437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:38.757498 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:38.758792 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.759420 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.760219 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.760545 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.760611 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:38.763424 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:38.763426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:38.763427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:38.763436 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:38.765142 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:38.765488 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:38.765489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:38.765489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:38.765543 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:38.765707 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:38.765708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:38.765708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:38.765827 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:38.765915 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:38.766031 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 22:00:38.766119 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260925 22:00:38.768162 11398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:38.768425 11398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:38.770334 11535 tablet_service.cc:1467] Processing CreateTablet for tablet 2fe6233c365040eba522d0f137331211 (DEFAULT_TABLE table=test-table [id=4670d97acddd4ea3bc33aa9e77d4fa43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:38.770385 11815 tablet_service.cc:1467] Processing CreateTablet for tablet 2fe6233c365040eba522d0f137331211 (DEFAULT_TABLE table=test-table [id=4670d97acddd4ea3bc33aa9e77d4fa43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:38.770524 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fe6233c365040eba522d0f137331211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:38.770558 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fe6233c365040eba522d0f137331211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:38.771908 11997 tablet_bootstrap.cc:492] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Bootstrap starting.
I20260925 22:00:38.771919 11998 tablet_bootstrap.cc:492] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Bootstrap starting.
I20260925 22:00:38.771968 11675 tablet_service.cc:1467] Processing CreateTablet for tablet 2fe6233c365040eba522d0f137331211 (DEFAULT_TABLE table=test-table [id=4670d97acddd4ea3bc33aa9e77d4fa43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:38.772100 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fe6233c365040eba522d0f137331211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:38.772303 11998 tablet_bootstrap.cc:654] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:38.772933 11997 tablet_bootstrap.cc:654] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:38.773042 11998 tablet_bootstrap.cc:492] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: No bootstrap required, opened a new log
I20260925 22:00:38.773106 11998 ts_tablet_manager.cc:1397] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:38.773244 11998 raft_consensus.cc:348] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.773332 11998 raft_consensus.cc:374] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:38.773455 11998 raft_consensus.cc:729] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Initialized, Role: FOLLOWER
I20260925 22:00:38.773506 11998 consensus_queue.cc:260] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.773775 11998 ts_tablet_manager.cc:1428] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:38.773854 12001 tablet_bootstrap.cc:492] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Bootstrap starting.
I20260925 22:00:38.773787 11997 tablet_bootstrap.cc:492] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: No bootstrap required, opened a new log
I20260925 22:00:38.774004 11997 ts_tablet_manager.cc:1397] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:38.774155 11997 raft_consensus.cc:348] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.774237 11997 raft_consensus.cc:374] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:38.774235 12001 tablet_bootstrap.cc:654] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:38.774264 11997 raft_consensus.cc:729] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c1f29642b54a7d870ad1a1f3800c44, State: Initialized, Role: FOLLOWER
I20260925 22:00:38.774327 11997 consensus_queue.cc:260] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.774541 11997 ts_tablet_manager.cc:1428] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:38.774914 12001 tablet_bootstrap.cc:492] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: No bootstrap required, opened a new log
I20260925 22:00:38.774973 12001 ts_tablet_manager.cc:1397] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:38.775108 12001 raft_consensus.cc:348] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.775185 12001 raft_consensus.cc:374] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:38.775205 12001 raft_consensus.cc:729] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Initialized, Role: FOLLOWER
I20260925 22:00:38.775262 12001 consensus_queue.cc:260] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.775358 12001 ts_tablet_manager.cc:1428] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:38.946126 12003 raft_consensus.cc:482] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:38.946264 12003 raft_consensus.cc:504] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.946408 12003 leader_election.cc:288] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:38.946828 11834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fe6233c365040eba522d0f137331211" candidate_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94" is_pre_election: true
I20260925 22:00:38.946805 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fe6233c365040eba522d0f137331211" candidate_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1" is_pre_election: true
I20260925 22:00:38.946928 11834 raft_consensus.cc:2454] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6c1f29642b54a7d870ad1a1f3800c44 in term 0.
I20260925 22:00:38.946945 11695 raft_consensus.cc:2454] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6c1f29642b54a7d870ad1a1f3800c44 in term 0.
I20260925 22:00:38.947170 11489 leader_election.cc:302] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, b6c1f29642b54a7d870ad1a1f3800c44; no voters:
I20260925 22:00:38.947304 12003 raft_consensus.cc:2781] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:38.947362 12003 raft_consensus.cc:482] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:38.947381 12003 raft_consensus.cc:3037] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:38.948174 12003 raft_consensus.cc:504] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.948315 12003 leader_election.cc:288] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [CANDIDATE]: Term 1 election: Requested vote from peers fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:38.948589 11695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fe6233c365040eba522d0f137331211" candidate_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa08a13531e948d9b56f58928bb828f1"
I20260925 22:00:38.948594 11834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fe6233c365040eba522d0f137331211" candidate_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94"
I20260925 22:00:38.948678 11695 raft_consensus.cc:3037] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:38.948678 11834 raft_consensus.cc:3037] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:38.949314 11834 raft_consensus.cc:2454] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6c1f29642b54a7d870ad1a1f3800c44 in term 1.
I20260925 22:00:38.949527 11489 leader_election.cc:302] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, b6c1f29642b54a7d870ad1a1f3800c44; no voters:
I20260925 22:00:38.949554 11695 raft_consensus.cc:2454] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6c1f29642b54a7d870ad1a1f3800c44 in term 1.
I20260925 22:00:38.949621 12003 raft_consensus.cc:2781] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:38.949890 12003 raft_consensus.cc:686] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 LEADER]: Becoming Leader. State: Replica: b6c1f29642b54a7d870ad1a1f3800c44, State: Running, Role: LEADER
I20260925 22:00:38.950008 12003 consensus_queue.cc:237] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.950855 11397 catalog_manager.cc:5495] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:38.961457 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:38.958717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.9.112.65:34085)
0925 22:00:38.958920 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:38.958921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:38.958927 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:00:38.958962 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:38.958962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:38.958983 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:38.959058 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:38.959216 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.959744 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.960634 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.960863 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.960924 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:38.961080 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:38.961197 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:38.961262 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:38.961299 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 22:00:38.961321 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":117,"threads_started":1}
I20260925 22:00:38.967586 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:38.964859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:45161 (local address 127.9.112.66:45817)
0925 22:00:38.965026 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:38.965029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:38.965037 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:38.965063 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:38.965064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:38.965076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:38.965147 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:38.965316 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.965851 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.966773 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.966999 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.967071 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:38.967271 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:38.967304 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:38.967342 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:38.967469 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 22:00:38.967496 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260925 22:00:38.967607 12010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:38.964725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.9.112.65:45161)
0925 22:00:38.964918 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:38.964931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:38.964988 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:38.965178 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:38.965179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:38.965182 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:38.965254 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:38.965255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.965981 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.965982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:38.966651 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:38.966653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.967117 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.967119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:38.967174 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:38.967194 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:38.967384 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:38.967387 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:38.967465 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 22:00:38.967523 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260925 22:00:38.970295 11834 raft_consensus.cc:1261] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Refusing update from remote peer b6c1f29642b54a7d870ad1a1f3800c44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:38.970355 11695 raft_consensus.cc:1261] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Refusing update from remote peer b6c1f29642b54a7d870ad1a1f3800c44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:38.970629 12004 consensus_queue.cc:1035] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:38.970803 12003 consensus_queue.cc:1035] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:38.979007 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:38.976681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:38365 (local address 127.9.112.65:34085)
0925 22:00:38.976731 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:38.976733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:38.976741 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:38.976797 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:38.976798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:38.976809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:38.976852 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:38.976975 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.977470 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.978225 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.978467 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.978533 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:38.978804 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:38.978842 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:38.978890 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:38.978923 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 22:00:38.978944 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:00:38.979035 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:38.976550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34085 (local address 127.9.112.66:38365)
0925 22:00:38.976683 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:38.976696 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:38.976746 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:38.976853 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:38.976854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:38.976855 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:38.976914 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:38.976916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.977558 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.977559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:38.978148 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:38.978150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:38.978576 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:38.978577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:38.978643 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:38.978649 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:38.978888 (+ 239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:38.978890 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:00:38.978919 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 22:00:38.978962 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":85,"threads_started":1}
I20260925 22:00:38.988855 11397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:38.988965 11397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:38.989634 11397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Sending DeleteTablet for 3 replicas of tablet 2fe6233c365040eba522d0f137331211
I20260925 22:00:38.989885 11815 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fe6233c365040eba522d0f137331211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:38 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:55400
I20260925 22:00:38.989909 11535 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fe6233c365040eba522d0f137331211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:38 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:54620
I20260925 22:00:38.989977 11988 tablet_replica.cc:330] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: stopping tablet replica
I20260925 22:00:38.989979 11675 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fe6233c365040eba522d0f137331211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:38 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:33550
I20260925 22:00:38.990025 11988 raft_consensus.cc:2229] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:38.990062 11988 raft_consensus.cc:2258] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:38.990083 11986 tablet_replica.cc:330] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: stopping tablet replica
I20260925 22:00:38.990137 11987 tablet_replica.cc:330] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: stopping tablet replica
I20260925 22:00:38.990149 11986 raft_consensus.cc:2229] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:38.990190 11987 raft_consensus.cc:2229] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:38.990222 11986 raft_consensus.cc:2258] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:38.990226 11987 raft_consensus.cc:2258] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:38.990558 11986 ts_tablet_manager.cc:1905] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:38.990558 11987 ts_tablet_manager.cc:1905] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:38.992177 11987 ts_tablet_manager.cc:1918] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:38.992234 11987 log.cc:1196] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/2fe6233c365040eba522d0f137331211
I20260925 22:00:38.992151 11397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 22:00:38.992296 11988 ts_tablet_manager.cc:1905] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 22:00:38.992322 11397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:38.992455 11987 ts_tablet_manager.cc:1939] T 2fe6233c365040eba522d0f137331211 P fa08a13531e948d9b56f58928bb828f1: Deleting consensus metadata
I20260925 22:00:38.992594 11986 ts_tablet_manager.cc:1918] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:38.992632 11986 log.cc:1196] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/2fe6233c365040eba522d0f137331211
I20260925 22:00:38.992794 11986 ts_tablet_manager.cc:1939] T 2fe6233c365040eba522d0f137331211 P b6c1f29642b54a7d870ad1a1f3800c44: Deleting consensus metadata
I20260925 22:00:38.993119 11382 catalog_manager.cc:4841] TS b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085): tablet 2fe6233c365040eba522d0f137331211 (table test-table [id=4670d97acddd4ea3bc33aa9e77d4fa43]) successfully deleted
I20260925 22:00:38.993412 11385 catalog_manager.cc:4841] TS fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817): tablet 2fe6233c365040eba522d0f137331211 (table test-table [id=4670d97acddd4ea3bc33aa9e77d4fa43]) successfully deleted
I20260925 22:00:38.993793 11988 ts_tablet_manager.cc:1918] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:00:38.993848 11988 log.cc:1196] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/2fe6233c365040eba522d0f137331211
I20260925 22:00:38.994029 11988 ts_tablet_manager.cc:1939] T 2fe6233c365040eba522d0f137331211 P ad2bd8211239458c81313ba9129e7b94: Deleting consensus metadata
I20260925 22:00:38.994045 11815 tablet_service.cc:1467] Processing CreateTablet for tablet 0dcd272e9a5e4e86b84fe588c87ae37a (DEFAULT_TABLE table=test-table [id=13a05bb5b7b94c398efa45a930403eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:38.994050 11535 tablet_service.cc:1467] Processing CreateTablet for tablet 0dcd272e9a5e4e86b84fe588c87ae37a (DEFAULT_TABLE table=test-table [id=13a05bb5b7b94c398efa45a930403eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:38.994131 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dcd272e9a5e4e86b84fe588c87ae37a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:38.994174 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dcd272e9a5e4e86b84fe588c87ae37a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:38.995026 11998 tablet_bootstrap.cc:492] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Bootstrap starting.
I20260925 22:00:38.995294 11998 tablet_bootstrap.cc:654] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:38.995414 11997 tablet_bootstrap.cc:492] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44: Bootstrap starting.
I20260925 22:00:38.995502 11385 catalog_manager.cc:4841] TS ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615): tablet 2fe6233c365040eba522d0f137331211 (table test-table [id=4670d97acddd4ea3bc33aa9e77d4fa43]) successfully deleted
I20260925 22:00:38.995745 11997 tablet_bootstrap.cc:654] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:38.995775 11675 tablet_service.cc:1467] Processing CreateTablet for tablet 0dcd272e9a5e4e86b84fe588c87ae37a (DEFAULT_TABLE table=test-table [id=13a05bb5b7b94c398efa45a930403eff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:38.995877 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dcd272e9a5e4e86b84fe588c87ae37a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:38.996269 11997 tablet_bootstrap.cc:492] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44: No bootstrap required, opened a new log
I20260925 22:00:38.996279 11998 tablet_bootstrap.cc:492] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: No bootstrap required, opened a new log
I20260925 22:00:38.996313 11998 ts_tablet_manager.cc:1397] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 22:00:38.996310 11997 ts_tablet_manager.cc:1397] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:38.996562 11997 raft_consensus.cc:348] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.996544 11998 raft_consensus.cc:348] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.996634 11998 raft_consensus.cc:374] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:38.996634 11997 raft_consensus.cc:374] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:38.996655 11997 raft_consensus.cc:729] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c1f29642b54a7d870ad1a1f3800c44, State: Initialized, Role: FOLLOWER
I20260925 22:00:38.996655 11998 raft_consensus.cc:729] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2bd8211239458c81313ba9129e7b94, State: Initialized, Role: FOLLOWER
I20260925 22:00:38.996697 11998 consensus_queue.cc:260] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.996697 11997 consensus_queue.cc:260] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.996802 11998 ts_tablet_manager.cc:1428] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:38.996857 11997 ts_tablet_manager.cc:1428] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:38.997242 12001 tablet_bootstrap.cc:492] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Bootstrap starting.
I20260925 22:00:38.997593 12001 tablet_bootstrap.cc:654] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:38.998414 12001 tablet_bootstrap.cc:492] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: No bootstrap required, opened a new log
I20260925 22:00:38.998469 12001 ts_tablet_manager.cc:1397] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:38.998566 12001 raft_consensus.cc:348] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.998633 12001 raft_consensus.cc:374] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:38.998646 12001 raft_consensus.cc:729] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Initialized, Role: FOLLOWER
I20260925 22:00:38.998687 12001 consensus_queue.cc:260] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:38.998780 12001 ts_tablet_manager.cc:1428] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:39.402972 11967 raft_consensus.cc:482] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:39.403057 11967 raft_consensus.cc:504] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:39.403205 11967 leader_election.cc:288] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:39.403471 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcd272e9a5e4e86b84fe588c87ae37a" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" is_pre_election: true
I20260925 22:00:39.403484 11834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcd272e9a5e4e86b84fe588c87ae37a" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94" is_pre_election: true
I20260925 22:00:39.403560 11834 raft_consensus.cc:2454] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 0.
I20260925 22:00:39.403565 11555 raft_consensus.cc:2454] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 0.
I20260925 22:00:39.403697 11629 leader_election.cc:302] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2bd8211239458c81313ba9129e7b94, fa08a13531e948d9b56f58928bb828f1; no voters:
I20260925 22:00:39.403810 11967 raft_consensus.cc:2781] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:39.403858 11967 raft_consensus.cc:482] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:39.403874 11967 raft_consensus.cc:3037] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:39.404635 11967 raft_consensus.cc:504] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:39.404801 11967 leader_election.cc:288] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 election: Requested vote from peers b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085), ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615)
I20260925 22:00:39.404955 11834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcd272e9a5e4e86b84fe588c87ae37a" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2bd8211239458c81313ba9129e7b94"
I20260925 22:00:39.404968 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcd272e9a5e4e86b84fe588c87ae37a" candidate_uuid: "fa08a13531e948d9b56f58928bb828f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c1f29642b54a7d870ad1a1f3800c44"
I20260925 22:00:39.405030 11834 raft_consensus.cc:3037] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:39.405030 11555 raft_consensus.cc:3037] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:39.405748 11834 raft_consensus.cc:2454] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 1.
I20260925 22:00:39.405745 11555 raft_consensus.cc:2454] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa08a13531e948d9b56f58928bb828f1 in term 1.
I20260925 22:00:39.405923 11626 leader_election.cc:302] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6c1f29642b54a7d870ad1a1f3800c44, fa08a13531e948d9b56f58928bb828f1; no voters:
I20260925 22:00:39.406025 11967 raft_consensus.cc:2781] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:39.406069 11967 raft_consensus.cc:686] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 1 LEADER]: Becoming Leader. State: Replica: fa08a13531e948d9b56f58928bb828f1, State: Running, Role: LEADER
I20260925 22:00:39.406107 11967 consensus_queue.cc:237] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } }
I20260925 22:00:39.406683 11397 catalog_manager.cc:5495] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa08a13531e948d9b56f58928bb828f1 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "fa08a13531e948d9b56f58928bb828f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa08a13531e948d9b56f58928bb828f1" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:39.445963 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:39.443750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.9.112.66:45817)
0925 22:00:39.443800 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:39.443801 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:39.443811 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:39.443876 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:39.443877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:39.443889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:39.443945 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:39.444077 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:39.444599 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:39.445325 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:39.445515 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:39.445577 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:39.445718 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:39.445755 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:39.445802 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:39.445857 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 22:00:39.445881 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:00:39.450230 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:39.448198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:39393 (local address 127.9.112.66:45817)
0925 22:00:39.448254 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:39.448255 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:39.448263 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:39.448281 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:39.448282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:39.448289 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:39.448382 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:39.448468 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:39.448962 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:39.449681 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:39.449871 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:39.449924 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:39.450026 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:39.450048 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:39.450080 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:39.450124 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 22:00:39.450140 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:00:39.450282 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:39.448071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45817 (local address 127.9.112.66:39393)
0925 22:00:39.448126 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:39.448136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:39.448173 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:39.448386 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:39.448386 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:39.448387 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:00:39.448421 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:39.448422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:39.449034 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:39.449035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:39.449588 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:39.449590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:39.449931 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:39.449932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:39.449970 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:39.449976 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:00:39.450086 (+ 110us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:00:39.450087 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:00:39.450116 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 22:00:39.450151 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 22:00:39.452174 11555 raft_consensus.cc:1261] T 0dcd272e9a5e4e86b84fe588c87ae37a P b6c1f29642b54a7d870ad1a1f3800c44 [term 1 FOLLOWER]: Refusing update from remote peer fa08a13531e948d9b56f58928bb828f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:39.452159 11834 raft_consensus.cc:1261] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Refusing update from remote peer fa08a13531e948d9b56f58928bb828f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:39.452395 11972 consensus_queue.cc:1035] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2bd8211239458c81313ba9129e7b94" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:39.452509 11963 consensus_queue.cc:1035] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6c1f29642b54a7d870ad1a1f3800c44" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 22:00:39.456310 11965 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 22:00:40.974581 11398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:40.974687 11398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:40.975502 11398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e95e17a6ffc44d33b30527e91295cbd0: Sending DeleteTablet for 3 replicas of tablet 0dcd272e9a5e4e86b84fe588c87ae37a
I20260925 22:00:40.975759 11815 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dcd272e9a5e4e86b84fe588c87ae37a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:40 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:55400
I20260925 22:00:40.975734 11674 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dcd272e9a5e4e86b84fe588c87ae37a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:40 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:33550
I20260925 22:00:40.975780 11535 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dcd272e9a5e4e86b84fe588c87ae37a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:40 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:54620
I20260925 22:00:40.975910 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 11470
I20260925 22:00:40.976851 12045 tablet_replica.cc:330] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: stopping tablet replica
I20260925 22:00:40.976924 12045 raft_consensus.cc:2229] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:40.977026 12045 raft_consensus.cc:2258] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:40.977293 12045 ts_tablet_manager.cc:1905] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:40.978979 12045 ts_tablet_manager.cc:1918] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:00:40.979056 12045 log.cc:1196] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/0dcd272e9a5e4e86b84fe588c87ae37a
I20260925 22:00:40.979280 12045 ts_tablet_manager.cc:1939] T 0dcd272e9a5e4e86b84fe588c87ae37a P fa08a13531e948d9b56f58928bb828f1: Deleting consensus metadata
I20260925 22:00:40.979892 11385 catalog_manager.cc:4841] TS fa08a13531e948d9b56f58928bb828f1 (127.9.112.66:45817): tablet 0dcd272e9a5e4e86b84fe588c87ae37a (table test-table [id=13a05bb5b7b94c398efa45a930403eff]) successfully deleted
I20260925 22:00:40.980471 12047 tablet_replica.cc:330] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: stopping tablet replica
I20260925 22:00:40.980543 12047 raft_consensus.cc:2229] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:40.980593 12047 raft_consensus.cc:2258] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:40.980857 12047 ts_tablet_manager.cc:1905] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:40.982132 12047 ts_tablet_manager.cc:1918] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 22:00:40.982182 12047 log.cc:1196] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/0dcd272e9a5e4e86b84fe588c87ae37a
I20260925 22:00:40.982352 12047 ts_tablet_manager.cc:1939] T 0dcd272e9a5e4e86b84fe588c87ae37a P ad2bd8211239458c81313ba9129e7b94: Deleting consensus metadata
W20260925 22:00:40.982939 11382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:34085 (error 108)
I20260925 22:00:40.983006 11385 catalog_manager.cc:4841] TS ad2bd8211239458c81313ba9129e7b94 (127.9.112.67:37615): tablet 0dcd272e9a5e4e86b84fe588c87ae37a (table test-table [id=13a05bb5b7b94c398efa45a930403eff]) successfully deleted
I20260925 22:00:40.983172 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 11610
I20260925 22:00:40.983944 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:40.983300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34085 (local address 127.0.0.1:54642)
0925 22:00:40.983853 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:40.983902 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":65,"threads_started":1}
W20260925 22:00:40.984030 11382 catalog_manager.cc:4584] TS b6c1f29642b54a7d870ad1a1f3800c44 (127.9.112.65:34085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0dcd272e9a5e4e86b84fe588c87ae37a: Network error: Client connection negotiation failed: client connection to 127.9.112.65:34085: connect: Connection refused (error 111)
I20260925 22:00:40.988481 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 11750
I20260925 22:00:40.993985 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 11366
2026-09-25T22:00:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6029 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:41 dist-test-slave-794d krb5kdc[12052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:41 dist-test-slave-794d krb5kdc[12052](info): set up 2 sockets
Sep 25 22:00:41 dist-test-slave-794d krb5kdc[12052](info): commencing operation
krb5kdc: starting...
W20260925 22:00:41.968101 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:42.087774 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35303
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:42.170420 12065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:42.170670 12065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:42.170696 12065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:42.171945 12065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:42.171998 12065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:42.172010 12065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:42.172021 12065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:42.172031 12065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:42.173628 12065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:42.173883 12065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:42.174111 12065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:42.179472 12070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.179613 12071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.179632 12073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:42.180063 12065 server_base.cc:1034] running on GCE node
I20260925 22:00:42.180538 12065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:42.180917 12065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:42.182099 12065 hybrid_clock.cc:648] HybridClock initialized: now 1790373642182078 us; error 36 us; skew 500 ppm
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:42.184923 12065 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:42.185417 12065 webserver.cc:460] Webserver started at http://127.9.112.126:36479/ using document root <none> and password file <none>
I20260925 22:00:42.185644 12065 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:42.185694 12065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:42.185823 12065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:42.186918 12065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "3753af3559b16368d91c85499db9ec83"
server_key_iv: "910569580fc015719fcde09551f3bef5"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.187276 12065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "3753af3559b16368d91c85499db9ec83"
server_key_iv: "910569580fc015719fcde09551f3bef5"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.188725 12065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:00:42.189752 12080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.190056 12065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:42.190168 12065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "3753af3559b16368d91c85499db9ec83"
server_key_iv: "910569580fc015719fcde09551f3bef5"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.190245 12065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:42.214727 12065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:42.217522 12065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:42.217684 12065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:42.221596 12065 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35303
I20260925 22:00:42.221791 12132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35303 every 8 connection(s)
I20260925 22:00:42.222066 12065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:42.222725 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12065
I20260925 22:00:42.222805 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:42.222908 12133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:42.222967 9665 external_mini_cluster.cc:1428] Setting key 1d79851f739b4942f336af63b793c6a9
I20260925 22:00:42.225761 12133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0: Bootstrap starting.
I20260925 22:00:42.226356 12133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:42.226605 12133 log.cc:824] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:42.227234 12133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0: No bootstrap required, opened a new log
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:42.229949 12133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35303 } }
I20260925 22:00:42.230171 12133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:42.230221 12133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38824b0bd7ff4b6ba561e44b6cefbcc0, State: Initialized, Role: FOLLOWER
I20260925 22:00:42.230366 12133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35303 } }
I20260925 22:00:42.230445 12133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:42.230480 12133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:42.230530 12133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:42.231312 12133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35303 } }
I20260925 22:00:42.231441 12133 leader_election.cc:302] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38824b0bd7ff4b6ba561e44b6cefbcc0; no voters:
I20260925 22:00:42.231638 12133 leader_election.cc:288] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:42.231777 12138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:42.231921 12138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [term 1 LEADER]: Becoming Leader. State: Replica: 38824b0bd7ff4b6ba561e44b6cefbcc0, State: Running, Role: LEADER
I20260925 22:00:42.231937 12133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:42.232124 12138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35303 } }
I20260925 22:00:42.233145 12139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38824b0bd7ff4b6ba561e44b6cefbcc0. Latest consensus state: current_term: 1 leader_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35303 } } }
I20260925 22:00:42.233261 12139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:42.233585 12151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:42.233773 12140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38824b0bd7ff4b6ba561e44b6cefbcc0" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35303 } } }
I20260925 22:00:42.233995 12140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:42.234082 12151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:42.234879 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.223500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.9.112.126:35303)
0925 22:00:42.223765 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:42.223774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:42.223802 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:00:42.224249 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:42.224256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:42.224307 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:42.224472 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:42.225096 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.226054 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.226864 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.227294 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.227413 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.230681 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:42.230686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:42.230687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:42.230702 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:42.232645 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.233145 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.233149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.233149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.233303 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.233545 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.233545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.233545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.233897 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:42.234538 (+ 641us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:42.234613 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 22:00:42.234723 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260925 22:00:42.235962 12151 catalog_manager.cc:1347] Generated new cluster ID: 2a0ba574df644cdd89e64189515abe69
I20260925 22:00:42.236012 12151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:42.242262 12151 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:42.243024 12151 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:42.249284 12151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38824b0bd7ff4b6ba561e44b6cefbcc0: Generated new TSK 0
I20260925 22:00:42.249513 12151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:42.279955 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35303
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:42.366492 12159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:42.366717 12159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:42.366742 12159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:42.367998 12159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:42.368053 12159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:42.368083 12159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:42.369665 12159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:42.369908 12159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:42.370141 12159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:00:42.375346 12159 server_base.cc:1034] running on GCE node
W20260925 22:00:42.375327 12164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.375298 12167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.375315 12165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:42.375707 12159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:42.375941 12159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:42.377095 12159 hybrid_clock.cc:648] HybridClock initialized: now 1790373642377076 us; error 34 us; skew 500 ppm
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:42.379541 12159 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:42.379976 12159 webserver.cc:460] Webserver started at http://127.9.112.65:35593/ using document root <none> and password file <none>
I20260925 22:00:42.380189 12159 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:42.380232 12159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:42.380383 12159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:42.381479 12159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "5d623808e25c4d6eb6186746d85710f4"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "b31d939e8371888de4c4647fac95b5de"
server_key_iv: "2a87f6d914fa33dce6771506c9a397ed"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.381805 12159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "5d623808e25c4d6eb6186746d85710f4"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "b31d939e8371888de4c4647fac95b5de"
server_key_iv: "2a87f6d914fa33dce6771506c9a397ed"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.383163 12159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:42.384069 12174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.384255 12159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:42.384327 12159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "5d623808e25c4d6eb6186746d85710f4"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "b31d939e8371888de4c4647fac95b5de"
server_key_iv: "2a87f6d914fa33dce6771506c9a397ed"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.384434 12159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:42.394608 12159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:42.396878 12159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:42.397001 12159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:42.397249 12159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:42.397558 12159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:42.397593 12159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.397615 12159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:42.397645 12159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.403337 12159 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:32823
I20260925 22:00:42.403391 12287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:32823 every 8 connection(s)
I20260925 22:00:42.403712 12159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:42.405489 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12159
I20260925 22:00:42.405589 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:42.405746 9665 external_mini_cluster.cc:1428] Setting key 9937b9b4a95ba2a7ceee4e5586bf9ff4
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
I20260925 22:00:42.415033 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.404855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:52475 (local address 127.9.112.126:35303)
0925 22:00:42.405004 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:42.405006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:42.405089 (+ 83us) server_negotiation.cc:407] Connection header received
0925 22:00:42.405788 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:42.405791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:42.405813 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:42.405945 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:42.407160 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.407776 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.408900 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.409103 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.409162 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.411703 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:42.411704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:42.411705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:42.411712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:42.413246 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.413849 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.413850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.413850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.413911 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.414208 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.414209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.414209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.414364 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:42.414434 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:42.414882 (+ 448us) server_negotiation.cc:299] Negotiation successful
0925 22:00:42.414960 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260925 22:00:42.415158 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.404781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35303 (local address 127.9.112.65:52475)
0925 22:00:42.404962 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:42.405006 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:42.405603 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:42.406016 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:42.406020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:42.406705 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:42.407016 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:42.407025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.407918 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.407919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:42.408814 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:42.408817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.409181 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.409181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:42.409305 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.409844 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:42.409853 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:42.411568 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:42.413346 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:42.413363 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:42.413367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:42.413692 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:42.413993 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:42.413994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:42.413994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:42.414121 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:42.414519 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:42.414528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:42.414697 (+ 169us) client_negotiation.cc:769] Sending connection context
0925 22:00:42.414807 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:00:42.414956 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260925 22:00:42.415757 12288 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35303
I20260925 22:00:42.415853 12288 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:42.416021 12288 heartbeater.cc:511] Master 127.9.112.126:35303 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
I20260925 22:00:42.416623 12097 ts_manager.cc:194] Registered new tserver with Master: 5d623808e25c4d6eb6186746d85710f4 (127.9.112.65:32823)
I20260925 22:00:42.417222 12097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:52475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:42.432919 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35303
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:42.515805 12293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:42.516036 12293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:42.516062 12293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:42.517297 12293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:42.517346 12293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:42.517372 12293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:42.518875 12293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:42.519114 12293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:42.519343 12293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:42.524266 12299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.524261 12298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.524266 12301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:42.524480 12293 server_base.cc:1034] running on GCE node
I20260925 22:00:42.524798 12293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:42.525025 12293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:42.526187 12293 hybrid_clock.cc:648] HybridClock initialized: now 1790373642526175 us; error 28 us; skew 500 ppm
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:42.528935 12293 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:42.529421 12293 webserver.cc:460] Webserver started at http://127.9.112.66:33869/ using document root <none> and password file <none>
I20260925 22:00:42.529631 12293 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:42.529673 12293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:42.529775 12293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:42.530812 12293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "648704a2c7bd4d79b69cb3176e0ae578"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "e62f78d5d663361b4ffded92517492eb"
server_key_iv: "51620eee73f450becb3daab19e1b6c31"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.531162 12293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "648704a2c7bd4d79b69cb3176e0ae578"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "e62f78d5d663361b4ffded92517492eb"
server_key_iv: "51620eee73f450becb3daab19e1b6c31"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.532531 12293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:00:42.533459 12308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.533643 12293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:42.533730 12293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "648704a2c7bd4d79b69cb3176e0ae578"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "e62f78d5d663361b4ffded92517492eb"
server_key_iv: "51620eee73f450becb3daab19e1b6c31"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.533804 12293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:42.557727 12293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:42.560021 12293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:42.560209 12293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:42.560472 12293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:42.560781 12293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:42.560817 12293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.560844 12293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:42.560868 12293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.566311 12293 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:36195
I20260925 22:00:42.566360 12421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:36195 every 8 connection(s)
I20260925 22:00:42.566668 12293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:42.567996 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12293
I20260925 22:00:42.568075 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:42.568252 9665 external_mini_cluster.cc:1428] Setting key cc0552fffc491c3165d7c7b87b5eb8c1
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
I20260925 22:00:42.577069 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.567402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:55131 (local address 127.9.112.126:35303)
0925 22:00:42.567459 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:42.567462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:42.567638 (+ 176us) server_negotiation.cc:407] Connection header received
0925 22:00:42.568232 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:42.568233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:42.568246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:42.568308 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:42.569398 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.570055 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.571568 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.571810 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.571873 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.574405 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:42.574407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:42.574407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:42.574415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:42.575713 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.576170 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.576170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.576171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.576226 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.576480 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.576480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.576481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.576581 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:42.576628 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:42.576901 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 22:00:42.576983 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:42.577188 12424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.567364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35303 (local address 127.9.112.66:55131)
0925 22:00:42.567506 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:42.567550 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:42.568058 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:42.568465 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:42.568469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:42.569091 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:42.569299 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:42.569304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.570160 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.570162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:42.571403 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:42.571407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.571897 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.571898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:42.572040 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.572607 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:42.572618 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:42.574240 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:42.575784 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:42.575790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:42.575793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:42.576080 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:42.576288 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:42.576289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:42.576290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:42.576435 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:42.576628 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:42.576631 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:42.576752 (+ 121us) client_negotiation.cc:769] Sending connection context
0925 22:00:42.576831 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:00:42.577004 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260925 22:00:42.577628 12422 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35303
I20260925 22:00:42.577728 12422 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:42.577913 12422 heartbeater.cc:511] Master 127.9.112.126:35303 requested a full tablet report, sending...
I20260925 22:00:42.578246 12097 ts_manager.cc:194] Registered new tserver with Master: 648704a2c7bd4d79b69cb3176e0ae578 (127.9.112.66:36195)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
I20260925 22:00:42.578651 12097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:55131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:42.595767 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35303
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:42.681180 12427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:42.681442 12427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:42.681480 12427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:42.682835 12427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:42.682893 12427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:42.682931 12427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:42.684576 12427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:42.684856 12427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:42.685086 12427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:42.690311 12432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.690349 12433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:42.690330 12435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:42.690658 12427 server_base.cc:1034] running on GCE node
I20260925 22:00:42.690881 12427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:42.691112 12427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:42.692323 12427 hybrid_clock.cc:648] HybridClock initialized: now 1790373642692296 us; error 26 us; skew 500 ppm
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:42.695103 12427 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:42.695540 12427 webserver.cc:460] Webserver started at http://127.9.112.67:34525/ using document root <none> and password file <none>
I20260925 22:00:42.695767 12427 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:42.695814 12427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:42.695925 12427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:42.697057 12427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "3e52e6796c074eb8af6d1850c6e8c6df"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "de66eb1ec1f820961bc3bb7be3dca109"
server_key_iv: "27631cadbab9b5fe1ed417622b73633b"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.697394 12427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "3e52e6796c074eb8af6d1850c6e8c6df"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "de66eb1ec1f820961bc3bb7be3dca109"
server_key_iv: "27631cadbab9b5fe1ed417622b73633b"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.698652 12427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:42.699455 12442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.699661 12427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:42.699751 12427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "3e52e6796c074eb8af6d1850c6e8c6df"
format_stamp: "Formatted at 2026-09-25 22:00:42 on dist-test-slave-794d"
server_key: "de66eb1ec1f820961bc3bb7be3dca109"
server_key_iv: "27631cadbab9b5fe1ed417622b73633b"
server_key_version: "encryptionkey@0"
I20260925 22:00:42.699826 12427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:42.727226 12427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:42.729305 12427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:42.729429 12427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:42.729656 12427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:42.729981 12427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:42.730016 12427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.730039 12427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:42.730063 12427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:42.735805 12427 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:33231
I20260925 22:00:42.735877 12555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:33231 every 8 connection(s)
I20260925 22:00:42.736160 12427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:42.741163 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12427
I20260925 22:00:42.741277 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:42.741446 9665 external_mini_cluster.cc:1428] Setting key f44cc134ebd20abc31e99151c9f68b23
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373642, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:42.745945 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.736901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:33189 (local address 127.9.112.126:35303)
0925 22:00:42.736971 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:42.736973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:42.737155 (+ 182us) server_negotiation.cc:407] Connection header received
0925 22:00:42.737693 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:42.737694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:42.737709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:42.737809 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:42.738788 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.739279 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.740308 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.740565 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.740622 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.742999 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:42.743001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:42.743001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:42.743008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:42.744483 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.744998 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.744999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.744999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.745062 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:42.745307 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:42.745308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:42.745308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:42.745435 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:42.745485 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:42.745762 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 22:00:42.745872 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:00:42.746099 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:42.736878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35303 (local address 127.9.112.67:33189)
0925 22:00:42.737033 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:42.737072 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:42.737572 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:42.737888 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:42.737893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:42.738483 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:42.738681 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:42.738686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.739365 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.739366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:42.740199 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:42.740202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:42.740660 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:42.740661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:42.740798 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:42.741343 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:42.741354 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:42.742896 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:42.744575 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:42.744582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:42.744584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:42.744894 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:42.745133 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:42.745133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:42.745134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:42.745251 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:42.745477 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:42.745480 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:42.745608 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 22:00:42.745743 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 22:00:42.745894 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260925 22:00:42.746675 12556 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35303
I20260925 22:00:42.746811 12556 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:42.746979 12556 heartbeater.cc:511] Master 127.9.112.126:35303 requested a full tablet report, sending...
I20260925 22:00:42.747409 12097 ts_manager.cc:194] Registered new tserver with Master: 3e52e6796c074eb8af6d1850c6e8c6df (127.9.112.67:33231)
I20260925 22:00:42.747862 12097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:33189
I20260925 22:00:42.754091 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 22:00:42.765369 12136 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:42.764494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.9.112.126:35303)
0925 22:00:42.764549 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:42.764552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:42.764645 (+ 93us) server_negotiation.cc:407] Connection header received
0925 22:00:42.764700 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:42.764700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:42.764710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:42.764778 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:42.765309 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43242: BlockingRecv error: recv got EOF from 127.0.0.1:43242 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:00:42.766445 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12159
I20260925 22:00:42.770242 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12293
I20260925 22:00:42.773833 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12427
I20260925 22:00:42.777668 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12065
2026-09-25T22:00:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1782 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12568](info): set up 2 sockets
Sep 25 22:00:42 dist-test-slave-794d krb5kdc[12568](info): commencing operation
krb5kdc: starting...
W20260925 22:00:43.752439 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:43 dist-test-slave-794d krb5kdc[12568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:43.876487 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44847
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:44847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:43.955276 12581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:43.955505 12581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:43.955528 12581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:43.956768 12581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:43.956809 12581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:43.956822 12581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:43.956835 12581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:43.956845 12581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:43.958365 12581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:44847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:43.958621 12581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:43.958840 12581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:43.963903 12589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:43.963912 12587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:43.964123 12581 server_base.cc:1034] running on GCE node
W20260925 22:00:43.963909 12586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:43.964509 12581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:43.964856 12581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:43.966025 12581 hybrid_clock.cc:648] HybridClock initialized: now 1790373643965984 us; error 57 us; skew 500 ppm
Sep 25 22:00:43 dist-test-slave-794d krb5kdc[12568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373643, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:43.968690 12581 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:43.969120 12581 webserver.cc:460] Webserver started at http://127.9.112.126:33271/ using document root <none> and password file <none>
I20260925 22:00:43.969311 12581 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:43.969355 12581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:43.969460 12581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:43.970525 12581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab"
format_stamp: "Formatted at 2026-09-25 22:00:43 on dist-test-slave-794d"
server_key: "11456cafaf01130d0514e687613a03f1"
server_key_iv: "4d0a70f896be401015fc676db64bc124"
server_key_version: "encryptionkey@0"
I20260925 22:00:43.970892 12581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab"
format_stamp: "Formatted at 2026-09-25 22:00:43 on dist-test-slave-794d"
server_key: "11456cafaf01130d0514e687613a03f1"
server_key_iv: "4d0a70f896be401015fc676db64bc124"
server_key_version: "encryptionkey@0"
I20260925 22:00:43.972307 12581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:43.973088 12596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:43.973237 12581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:43.973323 12581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab"
format_stamp: "Formatted at 2026-09-25 22:00:43 on dist-test-slave-794d"
server_key: "11456cafaf01130d0514e687613a03f1"
server_key_iv: "4d0a70f896be401015fc676db64bc124"
server_key_version: "encryptionkey@0"
I20260925 22:00:43.973387 12581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:43.982940 12581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:43.985071 12581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:43.985213 12581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:43.988832 12581 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:44847
I20260925 22:00:43.988873 12648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:44847 every 8 connection(s)
I20260925 22:00:43.989184 12581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:43.990118 12649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:43.991096 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12581
I20260925 22:00:43.991194 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:43.991366 9665 external_mini_cluster.cc:1428] Setting key 3b6f4685852b39272f3eccad4b1029db
I20260925 22:00:43.993027 12649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab: Bootstrap starting.
I20260925 22:00:43.993731 12649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:43.994028 12649 log.cc:824] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:43.994695 12649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab: No bootstrap required, opened a new log
I20260925 22:00:43.996618 12649 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44847 } }
I20260925 22:00:43.996759 12649 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:43.996797 12649 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5ea0cd42014cfba6da8f9f45dd98ab, State: Initialized, Role: FOLLOWER
I20260925 22:00:43.996964 12649 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44847 } }
I20260925 22:00:43.997066 12649 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:43.997107 12649 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:43.997155 12649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:43.997684 12649 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44847 } }
I20260925 22:00:43.997778 12649 leader_election.cc:302] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac5ea0cd42014cfba6da8f9f45dd98ab; no voters:
I20260925 22:00:43.997938 12649 leader_election.cc:288] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [CANDIDATE]: Term 1 election: Requested vote from peers
Sep 25 22:00:43 dist-test-slave-794d krb5kdc[12568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:43.997980 12654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:43.998128 12654 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [term 1 LEADER]: Becoming Leader. State: Replica: ac5ea0cd42014cfba6da8f9f45dd98ab, State: Running, Role: LEADER
I20260925 22:00:43.998188 12649 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:43.998230 12654 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44847 } }
I20260925 22:00:43.998574 12654 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac5ea0cd42014cfba6da8f9f45dd98ab. Latest consensus state: current_term: 1 leader_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44847 } } }
I20260925 22:00:43.998687 12654 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:43.998615 12656 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5ea0cd42014cfba6da8f9f45dd98ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44847 } } }
I20260925 22:00:43.998780 12656 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab [sys.catalog]: This master's current role is: LEADER
W20260925 22:00:43.999838 12670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:00:43.999894 12670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:00:44.000028 12662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:44.000249 12662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:44.002106 12662 catalog_manager.cc:1347] Generated new cluster ID: 76393dc572a34ee286f6041bc6e08e20
I20260925 22:00:44.002148 12662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:44.002825 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:43.991963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.9.112.126:44847)
0925 22:00:43.992253 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:43.992262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:43.992290 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:00:43.992877 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:43.992885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:43.992941 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:43.993080 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:43.993708 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:43.994705 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:43.995435 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:43.995909 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:43.996015 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:43.998555 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:43.998558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:43.998559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:43.998571 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:44.000936 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:44.001344 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:44.001347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:44.001348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:44.001468 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:44.001683 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:44.001683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:44.001684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:44.001969 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:44.002064 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:44.002466 (+ 402us) server_negotiation.cc:299] Negotiation successful
0925 22:00:44.002609 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":91,"threads_started":1}
I20260925 22:00:44.003744 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:00:44.019233 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:44.015168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.9.112.126:44847)
0925 22:00:44.015212 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:44.015214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:44.015306 (+ 92us) server_negotiation.cc:407] Connection header received
0925 22:00:44.015356 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:44.015356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:44.015367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:44.015440 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:44.015808 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:44.016636 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:44.017600 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:44.017941 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:44.018006 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:44.018726 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:44.018727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:44.018727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:44.018962 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:44.018983 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:44.018990 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:44.019049 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:44.019098 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 22:00:44.019127 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":22}
I20260925 22:00:44.020632 12662 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:44.021334 12662 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:44.027208 12662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac5ea0cd42014cfba6da8f9f45dd98ab: Generated new TSK 0
I20260925 22:00:44.027427 12662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:00:44.029786 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12581
2026-09-25T22:00:44Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1250 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:44 dist-test-slave-794d krb5kdc[12682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:44 dist-test-slave-794d krb5kdc[12682](info): set up 2 sockets
Sep 25 22:00:44 dist-test-slave-794d krb5kdc[12682](info): commencing operation
krb5kdc: starting...
W20260925 22:00:44.992177 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:45.112974 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:42005
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:45.195349 12695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:45.195577 12695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:45.195600 12695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:45.196770 12695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:45.196806 12695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:45.196818 12695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:45.196830 12695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:45.196841 12695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:45.198309 12695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:42005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:45.198515 12695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:45.198711 12695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:45.203754 12700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.203754 12701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:45.203953 12695 server_base.cc:1034] running on GCE node
W20260925 22:00:45.203787 12703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:45.204236 12695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:45.204584 12695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:45.205791 12695 hybrid_clock.cc:648] HybridClock initialized: now 1790373645205769 us; error 36 us; skew 500 ppm
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:45.208326 12695 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:45.208851 12695 webserver.cc:460] Webserver started at http://127.9.112.126:46293/ using document root <none> and password file <none>
I20260925 22:00:45.209062 12695 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:45.209113 12695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:45.209266 12695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:45.210333 12695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "e5abe9f00a7c4f418a8312ac5c921c69"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "3cc3d6e39dd304c69d18afd4784c5912"
server_key_iv: "4a0842c291a1d94f00de7f7edd30fbd2"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.210654 12695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "e5abe9f00a7c4f418a8312ac5c921c69"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "3cc3d6e39dd304c69d18afd4784c5912"
server_key_iv: "4a0842c291a1d94f00de7f7edd30fbd2"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.211930 12695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 22:00:45.212747 12710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.212917 12695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:00:45.213006 12695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "e5abe9f00a7c4f418a8312ac5c921c69"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "3cc3d6e39dd304c69d18afd4784c5912"
server_key_iv: "4a0842c291a1d94f00de7f7edd30fbd2"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.213065 12695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:45.228258 12695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:45.230558 12695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:45.230726 12695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:45.234608 12695 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:42005
I20260925 22:00:45.234653 12762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:42005 every 8 connection(s)
I20260925 22:00:45.234979 12695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:45.235958 12763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:45.238173 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12695
I20260925 22:00:45.238289 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:45.238454 9665 external_mini_cluster.cc:1428] Setting key 16e9fcc9b7f92eecb73285fe52667338
I20260925 22:00:45.238498 12763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69: Bootstrap starting.
I20260925 22:00:45.239183 12763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:45.239462 12763 log.cc:824] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:45.240183 12763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69: No bootstrap required, opened a new log
I20260925 22:00:45.242391 12763 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42005 } }
I20260925 22:00:45.242571 12763 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:45.242601 12763 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5abe9f00a7c4f418a8312ac5c921c69, State: Initialized, Role: FOLLOWER
I20260925 22:00:45.242728 12763 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42005 } }
I20260925 22:00:45.242815 12763 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:45.242899 12763 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:45.242944 12763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:45.243562 12763 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42005 } }
I20260925 22:00:45.243949 12763 leader_election.cc:302] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5abe9f00a7c4f418a8312ac5c921c69; no voters:
I20260925 22:00:45.244153 12763 leader_election.cc:288] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:45.244196 12768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:45.244426 12768 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [term 1 LEADER]: Becoming Leader. State: Replica: e5abe9f00a7c4f418a8312ac5c921c69, State: Running, Role: LEADER
I20260925 22:00:45.244475 12763 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:45.244534 12768 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42005 } }
I20260925 22:00:45.244918 12768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5abe9f00a7c4f418a8312ac5c921c69. Latest consensus state: current_term: 1 leader_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42005 } } }
I20260925 22:00:45.245007 12769 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5abe9f00a7c4f418a8312ac5c921c69" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42005 } } }
I20260925 22:00:45.245044 12768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:45.245059 12769 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:45.245250 12774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:45.245939 12774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:45.247828 12774 catalog_manager.cc:1347] Generated new cluster ID: 7f932062c2a445cfae59fcb96bad4303
I20260925 22:00:45.247877 12774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:45.249990 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.239019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.9.112.126:42005)
0925 22:00:45.239198 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:45.239208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:45.239242 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:00:45.239792 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:45.239803 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:45.239889 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:45.240018 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:45.240827 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.241820 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.242863 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.243362 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.243450 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.246280 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:45.246284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:45.246284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:45.246297 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:45.248395 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.248815 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.248818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.248819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.248924 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.249116 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.249116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.249117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.249391 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:45.249524 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:45.249631 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 22:00:45.249763 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260925 22:00:45.286094 12774 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:45.286964 12774 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:45.297035 12774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5abe9f00a7c4f418a8312ac5c921c69: Generated new TSK 0
I20260925 22:00:45.297274 12774 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:45.328634 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42005
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:45.415665 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:45.415987 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:45.416028 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:45.417380 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:45.417435 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:45.417464 12789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:45.419023 12789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:45.419243 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:45.419430 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:45.424593 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.424595 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.424609 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:45.424612 12789 server_base.cc:1034] running on GCE node
I20260925 22:00:45.425010 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:45.425316 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:45.426486 12789 hybrid_clock.cc:648] HybridClock initialized: now 1790373645426470 us; error 28 us; skew 500 ppm
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:45.429057 12789 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:45.429523 12789 webserver.cc:460] Webserver started at http://127.9.112.65:39337/ using document root <none> and password file <none>
I20260925 22:00:45.429731 12789 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:45.429780 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:45.429905 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:45.430953 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "003c6159566f4f5bb589859cf1fbf778"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "ec743820a43be81e8b594194d79e73da"
server_key_iv: "399da98812d46e2a8ed20057324aff04"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.431283 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "003c6159566f4f5bb589859cf1fbf778"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "ec743820a43be81e8b594194d79e73da"
server_key_iv: "399da98812d46e2a8ed20057324aff04"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.432797 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:45.433825 12804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.434087 12789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:45.434175 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "003c6159566f4f5bb589859cf1fbf778"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "ec743820a43be81e8b594194d79e73da"
server_key_iv: "399da98812d46e2a8ed20057324aff04"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.434247 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:45.458753 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:45.461011 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:45.461181 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:45.461438 12789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:45.461799 12789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:45.461838 12789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.461866 12789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:45.461884 12789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.467434 12789 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:46653
I20260925 22:00:45.467482 12917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:46653 every 8 connection(s)
I20260925 22:00:45.467787 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:45.474253 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12789
I20260925 22:00:45.474347 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:45.474520 9665 external_mini_cluster.cc:1428] Setting key c65e120a8e11c234a1736bbefdb459f0
I20260925 22:00:45.478173 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.468507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:49905 (local address 127.9.112.126:42005)
0925 22:00:45.468569 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:45.468570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:45.468830 (+ 260us) server_negotiation.cc:407] Connection header received
0925 22:00:45.469327 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:45.469329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:45.469341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:45.469394 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:45.470337 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.470835 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.471839 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.472060 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.472116 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.474872 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:45.474874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:45.474874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:45.474883 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:45.476542 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.477139 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.477141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.477141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.477205 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.477500 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.477501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.477501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.477625 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:45.477680 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:45.478020 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 22:00:45.478102 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:00:45.478389 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.468490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42005 (local address 127.9.112.65:49905)
0925 22:00:45.468649 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:45.468740 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:45.469226 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:45.469463 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:45.469466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:45.470033 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:45.470232 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:45.470237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.470925 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.470926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:45.471744 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:45.471746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.472138 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.472139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:45.472266 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.472903 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:45.472913 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:45.474715 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:45.476658 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:45.476671 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:45.476674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:45.477021 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:45.477279 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:45.477280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:45.477280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:45.477432 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:45.477688 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:45.477694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:45.477870 (+ 176us) client_negotiation.cc:769] Sending connection context
0925 22:00:45.477983 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 22:00:45.478161 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260925 22:00:45.478961 12918 heartbeater.cc:347] Connected to a master server at 127.9.112.126:42005
I20260925 22:00:45.479085 12918 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:45.479285 12918 heartbeater.cc:511] Master 127.9.112.126:42005 requested a full tablet report, sending...
I20260925 22:00:45.479928 12727 ts_manager.cc:194] Registered new tserver with Master: 003c6159566f4f5bb589859cf1fbf778 (127.9.112.65:46653)
I20260925 22:00:45.480734 12727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:49905
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:45.500890 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42005
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:45.582294 12923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:45.582532 12923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:45.582558 12923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:45.583809 12923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:45.583860 12923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:45.583889 12923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:45.585520 12923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:45.585752 12923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:45.585952 12923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:45.591169 12929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.591163 12931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.591162 12928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:45.591194 12923 server_base.cc:1034] running on GCE node
I20260925 22:00:45.591559 12923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:45.591873 12923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:45.593030 12923 hybrid_clock.cc:648] HybridClock initialized: now 1790373645593021 us; error 45 us; skew 500 ppm
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:45.595528 12923 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:45.595990 12923 webserver.cc:460] Webserver started at http://127.9.112.66:39253/ using document root <none> and password file <none>
I20260925 22:00:45.596215 12923 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:45.596263 12923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:45.596426 12923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:45.597496 12923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "19f700fd72e1469eb89bfce49eeeb3f1"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "b431a381730302cf46bf44f921e35b6b"
server_key_iv: "35534db11d47e912e680cc9bcc5051f3"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.597834 12923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "19f700fd72e1469eb89bfce49eeeb3f1"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "b431a381730302cf46bf44f921e35b6b"
server_key_iv: "35534db11d47e912e680cc9bcc5051f3"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.599186 12923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:45.599983 12938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.600194 12923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:45.600301 12923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "19f700fd72e1469eb89bfce49eeeb3f1"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "b431a381730302cf46bf44f921e35b6b"
server_key_iv: "35534db11d47e912e680cc9bcc5051f3"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.600474 12923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:45.617092 12923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:45.619154 12923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:45.619279 12923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:45.619472 12923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:45.619787 12923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:45.619822 12923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.619859 12923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:45.619879 12923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.626209 12923 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:46699
I20260925 22:00:45.626250 13051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:46699 every 8 connection(s)
I20260925 22:00:45.626602 12923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:45.636022 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 12923
I20260925 22:00:45.636127 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:45.636246 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.627464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:56741 (local address 127.9.112.126:42005)
0925 22:00:45.627547 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:45.627549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:45.627708 (+ 159us) server_negotiation.cc:407] Connection header received
0925 22:00:45.628223 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:45.628224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:45.628235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:45.628315 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:45.629284 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.629782 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.630896 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.631088 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.631147 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.633405 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:45.633407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:45.633407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:45.633414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:45.634830 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.635258 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.635260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.635260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.635320 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.635586 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.635587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.635587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.635712 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:45.635767 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:45.636072 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 22:00:45.636161 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 22:00:45.636379 9665 external_mini_cluster.cc:1428] Setting key 9e1b89ab592928e56c956ed30bc97141
I20260925 22:00:45.636494 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.627409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42005 (local address 127.9.112.66:56741)
0925 22:00:45.627576 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:45.627617 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:45.628110 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:45.628407 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:45.628411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:45.628989 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:45.629204 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:45.629209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.629885 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.629886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:45.630801 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:45.630804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.631188 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.631189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:45.631305 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.631828 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:45.631836 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:45.633302 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:45.634926 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:45.634932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:45.634934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:45.635169 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:45.635405 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:45.635406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:45.635407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:45.635521 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:45.635801 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:45.635807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:45.635940 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 22:00:45.636035 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 22:00:45.636209 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260925 22:00:45.636972 13052 heartbeater.cc:347] Connected to a master server at 127.9.112.126:42005
I20260925 22:00:45.637074 13052 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:45.637306 13052 heartbeater.cc:511] Master 127.9.112.126:42005 requested a full tablet report, sending...
I20260925 22:00:45.637679 12727 ts_manager.cc:194] Registered new tserver with Master: 19f700fd72e1469eb89bfce49eeeb3f1 (127.9.112.66:46699)
I20260925 22:00:45.638131 12727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:56741
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:45.664948 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42005
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:45.749598 13057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:45.749814 13057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:45.749841 13057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:45.751152 13057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:45.751212 13057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:45.751240 13057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:45.752765 13057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:45.752967 13057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:45.753156 13057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:45.758153 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.758144 13065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:45.758144 13063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:45.758438 13057 server_base.cc:1034] running on GCE node
I20260925 22:00:45.758639 13057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:45.758884 13057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:45.760035 13057 hybrid_clock.cc:648] HybridClock initialized: now 1790373645760019 us; error 31 us; skew 500 ppm
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:45.762619 13057 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:45.763100 13057 webserver.cc:460] Webserver started at http://127.9.112.67:42853/ using document root <none> and password file <none>
I20260925 22:00:45.763311 13057 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:45.763361 13057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:45.763489 13057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:45.764683 13057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "c61b04e86ac64a7cb8dae80809441cb9"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "f40f42d49dac56c04be4ea244835ae81"
server_key_iv: "412669ab23f432e72fb80a0536f21001"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.765051 13057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "c61b04e86ac64a7cb8dae80809441cb9"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "f40f42d49dac56c04be4ea244835ae81"
server_key_iv: "412669ab23f432e72fb80a0536f21001"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.766398 13057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:45.767166 13072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.767362 13057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:45.767465 13057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "c61b04e86ac64a7cb8dae80809441cb9"
format_stamp: "Formatted at 2026-09-25 22:00:45 on dist-test-slave-794d"
server_key: "f40f42d49dac56c04be4ea244835ae81"
server_key_iv: "412669ab23f432e72fb80a0536f21001"
server_key_version: "encryptionkey@0"
I20260925 22:00:45.767530 13057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:45.789239 13057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:45.791971 13057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:45.792148 13057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:45.792423 13057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:45.792742 13057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:45.792778 13057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.792816 13057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:45.792840 13057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:45.798815 13057 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:38385
I20260925 22:00:45.798869 13185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:38385 every 8 connection(s)
I20260925 22:00:45.799187 13057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:45.800258 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13057
I20260925 22:00:45.800375 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:45.800536 9665 external_mini_cluster.cc:1428] Setting key de2568feb7867cea61cec00e621f84ab
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:45.808529 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.799943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:43007 (local address 127.9.112.126:42005)
0925 22:00:45.799997 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:45.799998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:45.800180 (+ 182us) server_negotiation.cc:407] Connection header received
0925 22:00:45.800759 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:45.800761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:45.800775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:45.800838 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:45.801865 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.802370 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.803390 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.803581 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.803635 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.805862 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:45.805863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:45.805864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:45.805871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:45.807121 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.807529 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.807530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.807531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.807597 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.807899 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.807900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.807900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.808023 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:45.808091 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:45.808390 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 22:00:45.808459 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:00:45.808706 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.799888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42005 (local address 127.9.112.67:43007)
0925 22:00:45.800048 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:45.800092 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:45.800630 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:45.800910 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:45.800915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:45.801588 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:45.801787 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:45.801792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.802444 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.802444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:45.803284 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:45.803287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.803657 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.803658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:45.803772 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.804254 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:45.804283 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:45.805743 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:45.807209 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:45.807215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:45.807217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:45.807454 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:45.807705 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:45.807706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:45.807706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:45.807812 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:45.808089 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:45.808093 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:45.808220 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 22:00:45.808306 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 22:00:45.808472 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260925 22:00:45.809141 13186 heartbeater.cc:347] Connected to a master server at 127.9.112.126:42005
I20260925 22:00:45.809232 13186 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:45.809379 13186 heartbeater.cc:511] Master 127.9.112.126:42005 requested a full tablet report, sending...
I20260925 22:00:45.809752 12727 ts_manager.cc:194] Registered new tserver with Master: c61b04e86ac64a7cb8dae80809441cb9 (127.9.112.67:38385)
I20260925 22:00:45.810117 12727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:43007
I20260925 22:00:45.812363 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373645, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:45.835246 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:45.827683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.9.112.126:42005)
0925 22:00:45.827766 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:45.827768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:45.827877 (+ 109us) server_negotiation.cc:407] Connection header received
0925 22:00:45.827911 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:45.827912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:45.827923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:45.827992 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:45.828728 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.829270 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.830094 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:45.830315 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:45.830381 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:45.832899 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:45.832901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:45.832901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:45.832909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:45.834205 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.834560 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.834562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.834562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.834615 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:45.834799 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:45.834799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:45.834799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:45.834950 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:45.835023 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:45.835115 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 22:00:45.835177 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 22:00:45.835568 12727 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44902
I20260925 22:00:45.835983 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12789
I20260925 22:00:45.839534 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12923
I20260925 22:00:45.843463 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13057
I20260925 22:00:45.847143 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 12695
2026-09-25T22:00:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1818 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[13198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[13198](info): set up 2 sockets
Sep 25 22:00:45 dist-test-slave-794d krb5kdc[13198](info): commencing operation
krb5kdc: starting...
W20260925 22:00:46.808207 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:46 dist-test-slave-794d krb5kdc[13198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:46.935652 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44313
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:44313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:47.018195 13211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:47.018435 13211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:47.018464 13211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:47.019655 13211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:47.019698 13211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:47.019718 13211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:47.019734 13211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:47.019750 13211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:47.021353 13211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:44313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:47.021625 13211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:47.021835 13211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:47.026539 13219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:47.026587 13211 server_base.cc:1034] running on GCE node
W20260925 22:00:47.026679 13217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.026539 13216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:47.027084 13211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:47.027383 13211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:47.028527 13211 hybrid_clock.cc:648] HybridClock initialized: now 1790373647028509 us; error 32 us; skew 500 ppm
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:47.034463 13211 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:47.034996 13211 webserver.cc:460] Webserver started at http://127.9.112.126:40563/ using document root <none> and password file <none>
I20260925 22:00:47.035233 13211 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:47.035280 13211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:47.035416 13211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:47.036579 13211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "d55487058a164163b4a4b1d57bc97d95"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "40b9f72008dd2b49f4f876727657c7d3"
server_key_iv: "1debfe44de7ef353b161c59c801bf385"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.036934 13211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "d55487058a164163b4a4b1d57bc97d95"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "40b9f72008dd2b49f4f876727657c7d3"
server_key_iv: "1debfe44de7ef353b161c59c801bf385"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.038234 13211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:47.039054 13226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.039212 13211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:47.039295 13211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "d55487058a164163b4a4b1d57bc97d95"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "40b9f72008dd2b49f4f876727657c7d3"
server_key_iv: "1debfe44de7ef353b161c59c801bf385"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.039361 13211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:47.057804 13211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:47.060072 13211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:47.060194 13211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:47.064051 13211 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:44313
I20260925 22:00:47.064095 13278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:44313 every 8 connection(s)
I20260925 22:00:47.064613 13211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:47.065274 13279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:47.067720 13279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95: Bootstrap starting.
I20260925 22:00:47.068277 13279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:47.068560 13279 log.cc:824] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:47.069235 13279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95: No bootstrap required, opened a new log
I20260925 22:00:47.070655 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13211
I20260925 22:00:47.070741 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:47.070886 9665 external_mini_cluster.cc:1428] Setting key 6a93dd0a22f70163ded25c585c7dedf9
I20260925 22:00:47.070822 13279 raft_consensus.cc:348] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55487058a164163b4a4b1d57bc97d95" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44313 } }
I20260925 22:00:47.070988 13279 raft_consensus.cc:374] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:47.071030 13279 raft_consensus.cc:729] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55487058a164163b4a4b1d57bc97d95, State: Initialized, Role: FOLLOWER
I20260925 22:00:47.071172 13279 consensus_queue.cc:260] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55487058a164163b4a4b1d57bc97d95" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44313 } }
I20260925 22:00:47.071247 13279 raft_consensus.cc:388] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:47.071276 13279 raft_consensus.cc:482] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:47.071318 13279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:47.071939 13279 raft_consensus.cc:504] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55487058a164163b4a4b1d57bc97d95" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44313 } }
I20260925 22:00:47.072046 13279 leader_election.cc:302] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d55487058a164163b4a4b1d57bc97d95; no voters:
I20260925 22:00:47.072191 13279 leader_election.cc:288] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:47.072216 13284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:47.072407 13284 raft_consensus.cc:686] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [term 1 LEADER]: Becoming Leader. State: Replica: d55487058a164163b4a4b1d57bc97d95, State: Running, Role: LEADER
I20260925 22:00:47.072464 13279 sys_catalog.cc:564] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:47.072504 13284 consensus_queue.cc:237] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55487058a164163b4a4b1d57bc97d95" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44313 } }
I20260925 22:00:47.072901 13285 sys_catalog.cc:455] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d55487058a164163b4a4b1d57bc97d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55487058a164163b4a4b1d57bc97d95" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44313 } } }
I20260925 22:00:47.072980 13285 sys_catalog.cc:458] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:47.073127 13286 sys_catalog.cc:455] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d55487058a164163b4a4b1d57bc97d95. Latest consensus state: current_term: 1 leader_uuid: "d55487058a164163b4a4b1d57bc97d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55487058a164163b4a4b1d57bc97d95" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44313 } } }
I20260925 22:00:47.073213 13286 sys_catalog.cc:458] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:47.073335 13293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:47.073850 13293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:47.075362 13293 catalog_manager.cc:1347] Generated new cluster ID: 227547160e5f45d28873d168aafa748d
I20260925 22:00:47.075413 13293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:47.080744 13293 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:47.081492 13293 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:47.081720 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.071333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55416 (local address 127.9.112.126:44313)
0925 22:00:47.071923 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:47.071934 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:47.071971 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:00:47.072091 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:47.072099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:47.072161 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:47.072319 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:47.073232 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.074604 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.075573 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.076029 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.076135 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.078569 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:47.078574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:47.078574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:47.078586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:47.080082 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.080415 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.080419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.080419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.080546 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.080771 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.080772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.080772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.081021 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:47.081171 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:47.081285 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 22:00:47.081479 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":535,"thread_start_us":484,"threads_started":1}
I20260925 22:00:47.096979 13293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d55487058a164163b4a4b1d57bc97d95: Generated new TSK 0
I20260925 22:00:47.097224 13293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:47.132512 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44313
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:47.215747 13305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:47.215982 13305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:47.216007 13305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:47.217259 13305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:47.217305 13305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:47.217331 13305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:47.218937 13305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:47.219224 13305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:47.219520 13305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:47.224829 13310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.224848 13313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.224953 13311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:47.225109 13305 server_base.cc:1034] running on GCE node
I20260925 22:00:47.225500 13305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:47.225759 13305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:47.226899 13305 hybrid_clock.cc:648] HybridClock initialized: now 1790373647226886 us; error 30 us; skew 500 ppm
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:47.229508 13305 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:47.229934 13305 webserver.cc:460] Webserver started at http://127.9.112.65:40629/ using document root <none> and password file <none>
I20260925 22:00:47.230155 13305 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:47.230204 13305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:47.230336 13305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:47.231380 13305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "ccec08d5cc33483f8783f0b9b647d4b7"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "6f102b74e167b21dd331ab2deea79643"
server_key_iv: "69f12a33b9185c3c0560dd06aa9b90ce"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.231703 13305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "ccec08d5cc33483f8783f0b9b647d4b7"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "6f102b74e167b21dd331ab2deea79643"
server_key_iv: "69f12a33b9185c3c0560dd06aa9b90ce"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.233033 13305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:47.233820 13320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.233974 13305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:47.234064 13305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "ccec08d5cc33483f8783f0b9b647d4b7"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "6f102b74e167b21dd331ab2deea79643"
server_key_iv: "69f12a33b9185c3c0560dd06aa9b90ce"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.234126 13305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:47.243494 13305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:47.245597 13305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:47.245741 13305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:47.245965 13305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:47.246238 13305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:47.246272 13305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.246300 13305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:47.246317 13305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.251631 13305 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:44289
I20260925 22:00:47.251688 13433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:44289 every 8 connection(s)
I20260925 22:00:47.251984 13305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:47.257647 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13305
I20260925 22:00:47.257725 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:47.257869 9665 external_mini_cluster.cc:1428] Setting key 453a015ecb4d9837f91b8107c48dbc69
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:47.261052 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.252689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:54509 (local address 127.9.112.126:44313)
0925 22:00:47.252793 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:47.252796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:47.252932 (+ 136us) server_negotiation.cc:407] Connection header received
0925 22:00:47.253799 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:47.253800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:47.253812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:47.253879 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:47.254440 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.254940 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.255889 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.256098 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.256153 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.258420 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:47.258422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:47.258422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:47.258435 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:47.259772 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.260155 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.260156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.260156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.260206 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.260453 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.260453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.260454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.260565 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:47.260618 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:47.260884 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 22:00:47.260962 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 22:00:47.261229 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.252660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44313 (local address 127.9.112.65:54509)
0925 22:00:47.252817 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:47.252859 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:47.253703 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:47.253935 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:47.253938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:47.254173 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:47.254365 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:47.254369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.255013 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.255014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:47.255813 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:47.255815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.256199 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.256200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:47.256316 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.256889 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:47.256897 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:47.258270 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:47.259850 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:47.259857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:47.259860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:47.260090 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:47.260258 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:47.260258 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:47.260259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:47.260401 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:47.260616 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:47.260622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:47.260758 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 22:00:47.260856 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 22:00:47.261013 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260925 22:00:47.261714 13434 heartbeater.cc:347] Connected to a master server at 127.9.112.126:44313
I20260925 22:00:47.261807 13434 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:47.261982 13434 heartbeater.cc:511] Master 127.9.112.126:44313 requested a full tablet report, sending...
I20260925 22:00:47.262496 13243 ts_manager.cc:194] Registered new tserver with Master: ccec08d5cc33483f8783f0b9b647d4b7 (127.9.112.65:44289)
I20260925 22:00:47.263129 13243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:54509
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:47.285367 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44313
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:47.367600 13439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:47.367865 13439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:47.367905 13439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:47.369278 13439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:47.369340 13439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:47.369380 13439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:47.371098 13439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:47.371387 13439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:47.371615 13439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:47.376917 13444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.376981 13447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.376919 13445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:47.377170 13439 server_base.cc:1034] running on GCE node
I20260925 22:00:47.377418 13439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:47.377668 13439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:47.378839 13439 hybrid_clock.cc:648] HybridClock initialized: now 1790373647378814 us; error 40 us; skew 500 ppm
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:47.381584 13439 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:47.382069 13439 webserver.cc:460] Webserver started at http://127.9.112.66:39675/ using document root <none> and password file <none>
I20260925 22:00:47.382318 13439 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:47.382368 13439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:47.382496 13439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:47.383541 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "d9fe16a1910e406aad60ee660cd0b44a"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "1d4c01a97f8d1ffa1458c4de21cdae6b"
server_key_iv: "9e220de2facad2bcd69da0ae23f81a2a"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.383881 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "d9fe16a1910e406aad60ee660cd0b44a"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "1d4c01a97f8d1ffa1458c4de21cdae6b"
server_key_iv: "9e220de2facad2bcd69da0ae23f81a2a"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.385295 13439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:47.386179 13454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.386435 13439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:47.386529 13439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "d9fe16a1910e406aad60ee660cd0b44a"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "1d4c01a97f8d1ffa1458c4de21cdae6b"
server_key_iv: "9e220de2facad2bcd69da0ae23f81a2a"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.386602 13439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:47.402709 13439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:47.405285 13439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:47.405442 13439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:47.405680 13439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:47.405977 13439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:47.406010 13439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.406039 13439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:47.406064 13439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.411780 13439 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:43125
I20260925 22:00:47.411854 13567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:43125 every 8 connection(s)
I20260925 22:00:47.412154 13439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:47.420820 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13439
I20260925 22:00:47.420935 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:47.421119 9665 external_mini_cluster.cc:1428] Setting key 37662b8355a735d03e72eef40be78441
I20260925 22:00:47.422170 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.413019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:44441 (local address 127.9.112.126:44313)
0925 22:00:47.413072 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:47.413074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:47.413177 (+ 103us) server_negotiation.cc:407] Connection header received
0925 22:00:47.414104 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:47.414105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:47.414115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:47.414171 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:47.414790 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.415321 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.416556 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.416755 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.416820 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.418975 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:47.418978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:47.418978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:47.418986 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:47.420324 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.421019 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.421021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.421022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.421091 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.421431 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.421432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.421432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.421578 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:47.421640 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:47.422026 (+ 386us) server_negotiation.cc:299] Negotiation successful
0925 22:00:47.422105 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:00:47.422423 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.412899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44313 (local address 127.9.112.66:44441)
0925 22:00:47.413070 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:47.413109 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:47.413998 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:47.414241 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:47.414244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:47.414500 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:47.414695 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:47.414699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.415426 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.415427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:47.416468 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:47.416471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.416837 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.416838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:47.416950 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.417485 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:47.417493 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:47.418864 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:47.420495 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:47.420504 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:47.420508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:47.420907 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:47.421164 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:47.421164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:47.421165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:47.421330 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:47.421647 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:47.421654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:47.421884 (+ 230us) client_negotiation.cc:769] Sending connection context
0925 22:00:47.422001 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 22:00:47.422191 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260925 22:00:47.422941 13568 heartbeater.cc:347] Connected to a master server at 127.9.112.126:44313
I20260925 22:00:47.423043 13568 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:47.423231 13568 heartbeater.cc:511] Master 127.9.112.126:44313 requested a full tablet report, sending...
I20260925 22:00:47.423655 13243 ts_manager.cc:194] Registered new tserver with Master: d9fe16a1910e406aad60ee660cd0b44a (127.9.112.66:43125)
I20260925 22:00:47.424222 13243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:44441
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:47.448278 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44313
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:47.533237 13573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:47.533458 13573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:47.533484 13573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:47.534708 13573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:47.534754 13573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:47.534786 13573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:47.536418 13573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:44313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:47.536659 13573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:47.536897 13573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:47.542177 13581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.542184 13579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:47.542878 13578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:47.543016 13573 server_base.cc:1034] running on GCE node
I20260925 22:00:47.543326 13573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:47.543597 13573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:47.544772 13573 hybrid_clock.cc:648] HybridClock initialized: now 1790373647544757 us; error 33 us; skew 500 ppm
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:47.547510 13573 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:47.548014 13573 webserver.cc:460] Webserver started at http://127.9.112.67:45967/ using document root <none> and password file <none>
I20260925 22:00:47.548220 13573 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:47.548267 13573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:47.548431 13573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:47.549465 13573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "1590a91385e349858d17f16aeebd5ade"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "d800e81e5275f1b6912a2c9c43badd86"
server_key_iv: "2a3ceea4ea01519992ee50e3ce6e290d"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.549786 13573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "1590a91385e349858d17f16aeebd5ade"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "d800e81e5275f1b6912a2c9c43badd86"
server_key_iv: "2a3ceea4ea01519992ee50e3ce6e290d"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.551383 13573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:47.552261 13588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.552500 13573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:47.552604 13573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "1590a91385e349858d17f16aeebd5ade"
format_stamp: "Formatted at 2026-09-25 22:00:47 on dist-test-slave-794d"
server_key: "d800e81e5275f1b6912a2c9c43badd86"
server_key_iv: "2a3ceea4ea01519992ee50e3ce6e290d"
server_key_version: "encryptionkey@0"
I20260925 22:00:47.552726 13573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:47.574061 13573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:47.576506 13573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:47.576635 13573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:47.576876 13573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:47.577179 13573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:47.577210 13573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.577234 13573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:47.577257 13573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:47.583025 13573 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:41049
I20260925 22:00:47.583073 13701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:41049 every 8 connection(s)
I20260925 22:00:47.583432 13573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:47.592911 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.584148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:59465 (local address 127.9.112.126:44313)
0925 22:00:47.584229 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:47.584231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:47.584398 (+ 167us) server_negotiation.cc:407] Connection header received
0925 22:00:47.585383 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:47.585384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:47.585395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:47.585461 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:47.586095 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.586622 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.587695 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.587900 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.587964 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.590266 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:47.590267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:47.590267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:47.590274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:47.591585 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.592007 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.592008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.592009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.592069 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.592279 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.592280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.592280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.592433 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:47.592485 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:47.592757 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 22:00:47.592828 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:00:47.593039 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.584118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44313 (local address 127.9.112.67:59465)
0925 22:00:47.584253 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:47.584289 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:47.585259 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:47.585525 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:47.585529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:47.585783 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:47.586017 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:47.586021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.586714 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.586716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:47.587595 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:47.587598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.587980 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.587981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:47.588090 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.588644 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:47.588652 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:47.590170 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:47.591671 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:47.591677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:47.591679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:47.591923 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:47.592136 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:47.592136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:47.592136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:47.592234 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:47.592482 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:47.592486 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:47.592608 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 22:00:47.592698 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:00:47.592839 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260925 22:00:47.593510 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13573
I20260925 22:00:47.593590 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:47.593559 13702 heartbeater.cc:347] Connected to a master server at 127.9.112.126:44313
I20260925 22:00:47.593668 13702 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:47.593770 9665 external_mini_cluster.cc:1428] Setting key f22ac234785fdb9cbb0006b66990f7ac
I20260925 22:00:47.593909 13702 heartbeater.cc:511] Master 127.9.112.126:44313 requested a full tablet report, sending...
I20260925 22:00:47.594283 13243 ts_manager.cc:194] Registered new tserver with Master: 1590a91385e349858d17f16aeebd5ade (127.9.112.67:41049)
I20260925 22:00:47.594624 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:47.594731 13243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:59465
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:47.617585 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.609897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.9.112.65:44289)
0925 22:00:47.610057 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:47.610062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:47.610071 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:47.610094 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:47.610096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:47.610109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:47.610175 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:47.610812 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.611527 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.612262 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.612642 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.612703 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.615004 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:47.615012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:47.615013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:47.615020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:47.616500 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.616813 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.616816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.616817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.616874 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.617037 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.617038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.617038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.617174 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:47.617313 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:47.617390 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 22:00:47.617469 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:47.625618 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:47.618753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55432 (local address 127.9.112.126:44313)
0925 22:00:47.618825 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:47.618826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:47.618864 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:00:47.618936 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:47.618937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:47.618947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:47.618991 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:47.619444 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.619920 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.620630 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:47.620818 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:47.620873 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:47.623009 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:47.623010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:47.623011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:47.623018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:47.624390 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.624723 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.624725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.624725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.624788 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:47.625024 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:47.625025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:47.625026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:47.625189 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:47.625278 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:47.625447 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 22:00:47.625525 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260925 22:00:47.625900 13243 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432
I20260925 22:00:47.626246 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13305
I20260925 22:00:47.630313 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13439
I20260925 22:00:47.634217 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13573
I20260925 22:00:47.638046 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13211
2026-09-25T22:00:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1790 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13714](info): set up 2 sockets
Sep 25 22:00:47 dist-test-slave-794d krb5kdc[13714](info): commencing operation
krb5kdc: starting...
W20260925 22:00:48.608132 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:48 dist-test-slave-794d krb5kdc[13714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:48.730937 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:39223
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:39223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:48.810593 13727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:48.810825 13727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:48.810847 13727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:48.812023 13727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:48.812059 13727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:48.812072 13727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:48.812083 13727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:48.812093 13727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:48.813776 13727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:39223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:39223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:48.813994 13727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:48.814194 13727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:48.818959 13735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:48.819005 13727 server_base.cc:1034] running on GCE node
W20260925 22:00:48.818944 13733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:48.818945 13732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:48.819383 13727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:48.819639 13727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:48.820788 13727 hybrid_clock.cc:648] HybridClock initialized: now 1790373648820754 us; error 47 us; skew 500 ppm
Sep 25 22:00:48 dist-test-slave-794d krb5kdc[13714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373648, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:48.823338 13727 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:48.823690 13727 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:48.823745 13727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:48.823870 13727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:48.825011 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "40e449fff1c044b282a4d7ce4fdd357d"
format_stamp: "Formatted at 2026-09-25 22:00:48 on dist-test-slave-794d"
server_key: "3ef9b9f41734e1b322c578a65a3b2394"
server_key_iv: "88037cff8ba2bd26ee12b7d3d8b8aed5"
server_key_version: "encryptionkey@0"
I20260925 22:00:48.825385 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "40e449fff1c044b282a4d7ce4fdd357d"
format_stamp: "Formatted at 2026-09-25 22:00:48 on dist-test-slave-794d"
server_key: "3ef9b9f41734e1b322c578a65a3b2394"
server_key_iv: "88037cff8ba2bd26ee12b7d3d8b8aed5"
server_key_version: "encryptionkey@0"
I20260925 22:00:48.826666 13727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:48.827453 13741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:48.827627 13727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:48.827710 13727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "40e449fff1c044b282a4d7ce4fdd357d"
format_stamp: "Formatted at 2026-09-25 22:00:48 on dist-test-slave-794d"
server_key: "3ef9b9f41734e1b322c578a65a3b2394"
server_key_iv: "88037cff8ba2bd26ee12b7d3d8b8aed5"
server_key_version: "encryptionkey@0"
I20260925 22:00:48.827771 13727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:48.857842 13727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:48.861349 13727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:48.861817 13727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:48.865882 13727 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:39223
I20260925 22:00:48.865908 13793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:39223 every 8 connection(s)
I20260925 22:00:48.866215 13727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:48.867096 13794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:48.869544 13794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d: Bootstrap starting.
I20260925 22:00:48.870111 13794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:48.870368 13794 log.cc:824] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:48.870975 13794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d: No bootstrap required, opened a new log
I20260925 22:00:48.872567 13794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e449fff1c044b282a4d7ce4fdd357d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 39223 } }
I20260925 22:00:48.872710 13794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:48.872731 13794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e449fff1c044b282a4d7ce4fdd357d, State: Initialized, Role: FOLLOWER
I20260925 22:00:48.872843 13794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e449fff1c044b282a4d7ce4fdd357d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 39223 } }
I20260925 22:00:48.872916 13794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:48.872939 13794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:48.872965 13794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:48.873720 13794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e449fff1c044b282a4d7ce4fdd357d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 39223 } }
I20260925 22:00:48.873806 13794 leader_election.cc:302] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40e449fff1c044b282a4d7ce4fdd357d; no voters:
I20260925 22:00:48.873934 13794 leader_election.cc:288] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:48.873999 13797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:48.874161 13794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:48.874202 13797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [term 1 LEADER]: Becoming Leader. State: Replica: 40e449fff1c044b282a4d7ce4fdd357d, State: Running, Role: LEADER
I20260925 22:00:48.874328 13797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e449fff1c044b282a4d7ce4fdd357d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 39223 } }
I20260925 22:00:48.874665 13798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40e449fff1c044b282a4d7ce4fdd357d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e449fff1c044b282a4d7ce4fdd357d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 39223 } } }
I20260925 22:00:48.874743 13798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:48.874917 13799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40e449fff1c044b282a4d7ce4fdd357d. Latest consensus state: current_term: 1 leader_uuid: "40e449fff1c044b282a4d7ce4fdd357d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e449fff1c044b282a4d7ce4fdd357d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 39223 } } }
I20260925 22:00:48.874977 13799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:48.875262 13806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:48.875674 13806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:48.876423 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13727
I20260925 22:00:48.876514 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:48.876688 9665 external_mini_cluster.cc:1428] Setting key 14d393de3d1ecb9908ef528c701109be
I20260925 22:00:48.877202 13806 catalog_manager.cc:1347] Generated new cluster ID: d7e7847dbb654ceaa3eb393abe88d271
I20260925 22:00:48.877257 13806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:48 dist-test-slave-794d krb5kdc[13714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:48.899284 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:48.877349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.9.112.126:39223)
0925 22:00:48.877507 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:48.877517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:48.887235 (+ 9718us) server_negotiation.cc:407] Connection header received
0925 22:00:48.887417 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:48.887426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:48.887513 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:48.887658 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:48.888602 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:48.890079 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:48.891200 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:48.891878 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:48.891993 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:48.894794 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:48.894800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:48.894800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:48.894815 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:48.897201 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:48.897657 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:48.897661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:48.897661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:48.897805 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:48.898039 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:48.898040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:48.898040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:48.898488 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:48.898659 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:48.898750 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 22:00:48.899028 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260925 22:00:48.902271 13806 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:48.902887 13806 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:48.907905 13806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d: Generated new TSK 0
I20260925 22:00:48.908118 13806 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:48.937299 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:39223
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:49.019980 13820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.020227 13820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.020253 13820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.021559 13820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.021615 13820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.021646 13820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:49.023322 13820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:39223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.023573 13820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.023816 13820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:49.029212 13825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.029433 13820 server_base.cc:1034] running on GCE node
W20260925 22:00:49.029212 13826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.029219 13828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.029731 13820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:49.029999 13820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:49.031147 13820 hybrid_clock.cc:648] HybridClock initialized: now 1790373649031135 us; error 28 us; skew 500 ppm
Sep 25 22:00:49 dist-test-slave-794d krb5kdc[13714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373649, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:49.034186 13820 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:49.034549 13820 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:49.034598 13820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:49.034708 13820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:49.035779 13820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "bab1f6e75b9a4aa8bf60faf966a23564"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "e5ce052bc48b8a5ceb5139bd8d53de75"
server_key_iv: "a1e1d5b988074479320b3d377ed43b2c"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.036105 13820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "bab1f6e75b9a4aa8bf60faf966a23564"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "e5ce052bc48b8a5ceb5139bd8d53de75"
server_key_iv: "a1e1d5b988074479320b3d377ed43b2c"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.037535 13820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:49.038339 13834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.038525 13820 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:49.038617 13820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "bab1f6e75b9a4aa8bf60faf966a23564"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "e5ce052bc48b8a5ceb5139bd8d53de75"
server_key_iv: "a1e1d5b988074479320b3d377ed43b2c"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.038733 13820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:49.053874 13820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:49.056437 13820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:49.056594 13820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:49.056840 13820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:49.057142 13820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:49.057174 13820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.057204 13820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:49.057233 13820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.062896 13820 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:38735
I20260925 22:00:49.062978 13947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:38735 every 8 connection(s)
I20260925 22:00:49.063215 13820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:00:49 dist-test-slave-794d krb5kdc[13714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373649, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:49.072371 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.063962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:42351 (local address 127.9.112.126:39223)
0925 22:00:49.064053 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.064055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.064193 (+ 138us) server_negotiation.cc:407] Connection header received
0925 22:00:49.064753 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.064754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.064766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.064832 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.065881 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.066394 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.067395 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.067598 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.067655 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.069754 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.069756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.069756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:49.069764 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.071043 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.071426 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.071428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.071428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.071479 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.071690 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.071691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.071692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.071807 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.071863 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.072118 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.072216 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 22:00:49.072513 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.063870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:39223 (local address 127.9.112.65:42351)
0925 22:00:49.064056 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.064104 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.064635 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.064925 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.064928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.065196 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:49.065729 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.065735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.066482 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.066483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.067308 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.067311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.067668 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.067669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.067787 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.068291 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:49.068299 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:49.069665 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:49.071118 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:49.071124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:49.071127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:49.071360 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:49.071530 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:49.071531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:49.071531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:49.071637 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:49.071861 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:49.071867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:49.072004 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.072092 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.072245 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260925 22:00:49.072502 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13820
I20260925 22:00:49.072598 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:49.072788 9665 external_mini_cluster.cc:1428] Setting key cfe42f01eea1a076c17b1397a779f45f
I20260925 22:00:49.073205 13948 heartbeater.cc:347] Connected to a master server at 127.9.112.126:39223
I20260925 22:00:49.073307 13948 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:49.073452 13948 heartbeater.cc:511] Master 127.9.112.126:39223 requested a full tablet report, sending...
I20260925 22:00:49.073997 13758 ts_manager.cc:194] Registered new tserver with Master: bab1f6e75b9a4aa8bf60faf966a23564 (127.9.112.65:38735)
I20260925 22:00:49.074895 13758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:42351
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:49.102288 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:39223
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:49.185940 13953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.186177 13953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.186213 13953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.187680 13953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.187741 13953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.187794 13953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:49.189515 13953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:39223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.189786 13953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.190016 13953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:49.195353 13961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.195353 13959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.195353 13958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.195554 13953 server_base.cc:1034] running on GCE node
I20260925 22:00:49.195889 13953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:49.196167 13953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:49.197305 13953 hybrid_clock.cc:648] HybridClock initialized: now 1790373649197288 us; error 32 us; skew 500 ppm
Sep 25 22:00:49 dist-test-slave-794d krb5kdc[13714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373649, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:49.199723 13953 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:49.200040 13953 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:49.200083 13953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:49.200182 13953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:49.201243 13953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "e149c6de27ba435b853eb40fff6f0bbe"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "5672dbbdcaa203116762f37474ef7858"
server_key_iv: "52aa4849885c90fcf2cc83c0dfba7082"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.201558 13953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "e149c6de27ba435b853eb40fff6f0bbe"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "5672dbbdcaa203116762f37474ef7858"
server_key_iv: "52aa4849885c90fcf2cc83c0dfba7082"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.202673 13953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:49.203323 13967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.203500 13953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:49.203580 13953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "e149c6de27ba435b853eb40fff6f0bbe"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "5672dbbdcaa203116762f37474ef7858"
server_key_iv: "52aa4849885c90fcf2cc83c0dfba7082"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.203644 13953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:49.228060 13953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:49.230345 13953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:49.230490 13953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:49.230731 13953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:49.231061 13953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:49.231098 13953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.231127 13953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:49.231150 13953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.237421 13953 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:38797
I20260925 22:00:49.237453 14080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:38797 every 8 connection(s)
I20260925 22:00:49.237739 13953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:00:49 dist-test-slave-794d krb5kdc[13714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373649, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:49.247596 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 13953
I20260925 22:00:49.247691 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:49.247874 9665 external_mini_cluster.cc:1428] Setting key 7c58f197e088293b4d48d95e5ec55272
I20260925 22:00:49.247936 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.238524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:52979 (local address 127.9.112.126:39223)
0925 22:00:49.238678 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.238680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.238791 (+ 111us) server_negotiation.cc:407] Connection header received
0925 22:00:49.239296 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.239297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.239309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.239369 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.240450 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.241016 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.242121 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.242348 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.242411 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.244870 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.244872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.244872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:49.244880 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.246489 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.246942 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.246944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.246944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.247007 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.247222 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.247222 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.247223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.247340 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.247408 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.247752 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.247845 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260925 22:00:49.248106 14083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.238469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:39223 (local address 127.9.112.66:52979)
0925 22:00:49.238665 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.238704 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.239165 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.239483 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.239487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.239772 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:49.240317 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.240323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.241127 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.241128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.242039 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.242042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.242448 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.242449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.242569 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.243118 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:49.243126 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:49.244694 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:49.246586 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:49.246594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:49.246596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:49.246853 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:49.247067 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:49.247067 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:49.247067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:49.247168 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:49.247409 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:49.247415 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:49.247553 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.247641 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.247869 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":112,"threads_started":1}
I20260925 22:00:49.248603 14081 heartbeater.cc:347] Connected to a master server at 127.9.112.126:39223
I20260925 22:00:49.248708 14081 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:49.248859 14081 heartbeater.cc:511] Master 127.9.112.126:39223 requested a full tablet report, sending...
I20260925 22:00:49.249251 13757 ts_manager.cc:194] Registered new tserver with Master: e149c6de27ba435b853eb40fff6f0bbe (127.9.112.66:38797)
I20260925 22:00:49.249734 13757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:52979
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:49.276664 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:39223
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:49.362449 14086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.362738 14086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.362782 14086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.364089 14086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.364135 14086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.364172 14086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:49.365809 14086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:39223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.366055 14086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.366277 14086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:00:49.371716 14086 server_base.cc:1034] running on GCE node
W20260925 22:00:49.371656 14091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.371656 14092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.371637 14094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.372078 14086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:49.372375 14086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:49.373550 14086 hybrid_clock.cc:648] HybridClock initialized: now 1790373649373534 us; error 34 us; skew 500 ppm
Sep 25 22:00:49 dist-test-slave-794d krb5kdc[13714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373649, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:49.376410 14086 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:49.376755 14086 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:49.376803 14086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:49.376910 14086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:49.377976 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "4fdd541c08dd4897bd66ebb7b1a4d229"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "186424878d8c26d72ad4572f91c19f9a"
server_key_iv: "cfbfb371f72ec462b2f63dc15013b439"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.378314 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "4fdd541c08dd4897bd66ebb7b1a4d229"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "186424878d8c26d72ad4572f91c19f9a"
server_key_iv: "cfbfb371f72ec462b2f63dc15013b439"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.379848 14086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:49.380791 14100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.381045 14086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:00:49.381152 14086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "4fdd541c08dd4897bd66ebb7b1a4d229"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "186424878d8c26d72ad4572f91c19f9a"
server_key_iv: "cfbfb371f72ec462b2f63dc15013b439"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.381230 14086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:49.389559 14086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:49.392376 14086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:49.392508 14086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:49.392709 14086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:49.392988 14086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:49.393019 14086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.393041 14086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:49.393052 14086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.398595 14086 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:39313
I20260925 22:00:49.398669 14213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:39313 every 8 connection(s)
I20260925 22:00:49.398921 14086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:49.401577 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14086
I20260925 22:00:49.401671 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:49.401819 9665 external_mini_cluster.cc:1428] Setting key 324e0eada7a60cfd00fe7d05bbebb5b0
Sep 25 22:00:49 dist-test-slave-794d krb5kdc[13714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373649, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:49.408939 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.399624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:53469 (local address 127.9.112.126:39223)
0925 22:00:49.399675 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.399677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.399862 (+ 185us) server_negotiation.cc:407] Connection header received
0925 22:00:49.400324 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.400325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.400334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.400442 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.401646 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.402361 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.403462 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.403658 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.403725 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.406121 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.406122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.406122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:49.406130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.407495 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.407964 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.407965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.407966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.408026 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.408259 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.408260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.408260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.408432 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.408492 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.408764 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.408846 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:00:49.409152 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.399597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:39223 (local address 127.9.112.67:53469)
0925 22:00:49.399742 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.399783 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.400236 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.400633 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.400667 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.400940 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:49.401500 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.401508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.402458 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.402460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.403367 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.403370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.403742 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.403743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.403857 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.404368 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:49.404379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:49.406031 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:49.407574 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:49.407581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:49.407584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:49.407902 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:49.408082 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:49.408083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:49.408083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:49.408204 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:49.408489 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:49.408495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:49.408641 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.408737 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.408903 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
I20260925 22:00:49.409636 14214 heartbeater.cc:347] Connected to a master server at 127.9.112.126:39223
I20260925 22:00:49.409744 14214 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:49.409899 14214 heartbeater.cc:511] Master 127.9.112.126:39223 requested a full tablet report, sending...
I20260925 22:00:49.410276 13757 ts_manager.cc:194] Registered new tserver with Master: 4fdd541c08dd4897bd66ebb7b1a4d229 (127.9.112.67:39313)
I20260925 22:00:49.410645 13757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:53469
I20260925 22:00:49.413396 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:49.420565 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.414665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.9.112.126:39223)
0925 22:00:49.414719 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.414721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.414814 (+ 93us) server_negotiation.cc:407] Connection header received
0925 22:00:49.414840 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.414841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.414849 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.414896 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.415409 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.415895 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.416756 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.416951 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.417014 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.418123 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.418125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.418125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:49.418132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.419350 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.419662 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.419663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.419663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.419709 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:49.419910 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:49.419911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:49.419911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:49.420064 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.420145 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.420418 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.420495 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:00:49.422477 13757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:49.424065 13757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:49.431661 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.427144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.9.112.67:39313)
0925 22:00:49.427319 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.427323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.427333 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:49.427374 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.427376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.427391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.427472 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:49.427815 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.428828 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.430706 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.431253 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.431322 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.431385 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.431547 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.431593 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260925 22:00:49.431646 14228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.426958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:39313 (local address 127.0.0.1:34564)
0925 22:00:49.427147 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.427157 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.427202 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.427649 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.427650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.427651 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:49.427678 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.427679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.428940 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.428942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.430639 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.430642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.431416 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.431416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.431466 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.431469 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.431516 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.431547 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":79,"thread_start_us":57,"threads_started":1}
I20260925 22:00:49.431638 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.426881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.9.112.65:38735)
0925 22:00:49.427015 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.427021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.427363 (+ 342us) server_negotiation.cc:407] Connection header received
0925 22:00:49.427411 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.427413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.427430 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.427495 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:49.427705 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.428871 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.430480 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.431216 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.431320 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.431420 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.431496 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.431541 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260925 22:00:49.432044 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.426997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58040 (local address 127.9.112.66:38797)
0925 22:00:49.427329 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.427332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.427341 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:49.427374 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.427376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.427392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.427472 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:49.427630 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.428868 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.430247 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.431227 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.431296 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.431366 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.431920 (+ 554us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.431967 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":66,"threads_started":1}
I20260925 22:00:49.432366 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.426777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:38797 (local address 127.0.0.1:58040)
0925 22:00:49.427060 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.427092 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.427168 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.427485 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.427488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.427490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:49.427572 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.427574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.428969 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.428969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.430161 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.430165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.431581 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.431582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.431658 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.431661 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.431704 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.432199 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":68,"thread_start_us":86,"threads_started":1}
I20260925 22:00:49.432469 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.426776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:38735 (local address 127.0.0.1:35738)
0925 22:00:49.427049 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.427290 (+ 241us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.427344 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.427588 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.427589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.427591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:49.427632 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.427633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.429192 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.429192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.430311 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.430314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.431310 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.431311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.431394 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.431409 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.431484 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.431541 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260925 22:00:49.432613 14148 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2764512c1b4f66916890434c65eeb6 (DEFAULT_TABLE table=test-table [id=9ca65e5d36834b3eba690db7ebd5f19e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:49.432969 14148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2764512c1b4f66916890434c65eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:49.433023 14015 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2764512c1b4f66916890434c65eeb6 (DEFAULT_TABLE table=test-table [id=9ca65e5d36834b3eba690db7ebd5f19e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:49.433323 14015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2764512c1b4f66916890434c65eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:49.435528 14231 tablet_bootstrap.cc:492] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Bootstrap starting.
I20260925 22:00:49.435858 14232 tablet_bootstrap.cc:492] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Bootstrap starting.
I20260925 22:00:49.436194 14231 tablet_bootstrap.cc:654] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:49.436182 13882 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2764512c1b4f66916890434c65eeb6 (DEFAULT_TABLE table=test-table [id=9ca65e5d36834b3eba690db7ebd5f19e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:49.436504 14232 tablet_bootstrap.cc:654] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:49.436539 13882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2764512c1b4f66916890434c65eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:49.436805 14231 log.cc:824] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:49.436975 14232 log.cc:824] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:49.437520 14231 tablet_bootstrap.cc:492] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: No bootstrap required, opened a new log
I20260925 22:00:49.437593 14231 ts_tablet_manager.cc:1397] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:49.437697 14232 tablet_bootstrap.cc:492] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: No bootstrap required, opened a new log
I20260925 22:00:49.437753 14232 ts_tablet_manager.cc:1397] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:49.439302 14235 tablet_bootstrap.cc:492] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564: Bootstrap starting.
I20260925 22:00:49.439634 14231 raft_consensus.cc:348] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.439634 14232 raft_consensus.cc:348] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.439808 14231 raft_consensus.cc:374] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:49.439807 14232 raft_consensus.cc:374] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:49.439843 14231 raft_consensus.cc:729] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fdd541c08dd4897bd66ebb7b1a4d229, State: Initialized, Role: FOLLOWER
I20260925 22:00:49.439843 14232 raft_consensus.cc:729] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e149c6de27ba435b853eb40fff6f0bbe, State: Initialized, Role: FOLLOWER
I20260925 22:00:49.439911 14235 tablet_bootstrap.cc:654] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:49.439958 14231 consensus_queue.cc:260] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.439989 14232 consensus_queue.cc:260] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.440238 14231 ts_tablet_manager.cc:1428] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:49.440237 14232 ts_tablet_manager.cc:1428] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:00:49.440384 14081 heartbeater.cc:503] Master 127.9.112.126:39223 was elected leader, sending a full tablet report...
I20260925 22:00:49.440441 14214 heartbeater.cc:503] Master 127.9.112.126:39223 was elected leader, sending a full tablet report...
I20260925 22:00:49.440478 14235 log.cc:824] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:49.441396 14235 tablet_bootstrap.cc:492] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564: No bootstrap required, opened a new log
I20260925 22:00:49.441468 14235 ts_tablet_manager.cc:1397] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:49.442971 14235 raft_consensus.cc:348] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.443255 14235 raft_consensus.cc:374] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:49.443306 14235 raft_consensus.cc:729] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab1f6e75b9a4aa8bf60faf966a23564, State: Initialized, Role: FOLLOWER
I20260925 22:00:49.443406 14235 consensus_queue.cc:260] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.443689 13948 heartbeater.cc:503] Master 127.9.112.126:39223 was elected leader, sending a full tablet report...
I20260925 22:00:49.443702 14235 ts_tablet_manager.cc:1428] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:49.451759 14239 raft_consensus.cc:482] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:49.451894 14239 raft_consensus.cc:504] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.452224 14239 leader_election.cc:288] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e149c6de27ba435b853eb40fff6f0bbe (127.9.112.66:38797), 4fdd541c08dd4897bd66ebb7b1a4d229 (127.9.112.67:39313)
I20260925 22:00:49.455044 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.452429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:38797 (local address 127.9.112.65:49345)
0925 22:00:49.452471 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.452483 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.452535 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.452719 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.452720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.452721 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:49.452787 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.452789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.453482 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.453483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.454270 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.454272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.454797 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.454798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.454875 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.454878 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.454928 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.454964 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":20}
I20260925 22:00:49.455056 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.452452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:49345 (local address 127.9.112.66:38797)
0925 22:00:49.452515 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.452517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.452529 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:00:49.452617 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.452618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.452627 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.452668 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:49.452862 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.453405 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.454353 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.454714 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.454768 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.454775 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.454949 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.454987 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:00:49.455139 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.452591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:36021 (local address 127.9.112.67:39313)
0925 22:00:49.452632 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.452634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.452752 (+ 118us) server_negotiation.cc:407] Connection header received
0925 22:00:49.452792 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.452793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.452806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.452859 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:49.452989 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.453457 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.454377 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.454765 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.454826 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.454834 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.455039 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.455081 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:00:49.455135 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.452528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:39313 (local address 127.9.112.65:36021)
0925 22:00:49.452681 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.452697 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.452752 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.452869 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.452870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.452871 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:49.452932 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.452933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.453531 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.453532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.454298 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:49.454300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.454837 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.454838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:49.454980 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.454984 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.455033 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.455069 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":113,"threads_started":1}
I20260925 22:00:49.455404 14168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2764512c1b4f66916890434c65eeb6" candidate_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" is_pre_election: true
I20260925 22:00:49.455404 14035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2764512c1b4f66916890434c65eeb6" candidate_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e149c6de27ba435b853eb40fff6f0bbe" is_pre_election: true
I20260925 22:00:49.455559 14168 raft_consensus.cc:2454] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab1f6e75b9a4aa8bf60faf966a23564 in term 0.
I20260925 22:00:49.455560 14035 raft_consensus.cc:2454] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab1f6e75b9a4aa8bf60faf966a23564 in term 0.
I20260925 22:00:49.455777 13835 leader_election.cc:302] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fdd541c08dd4897bd66ebb7b1a4d229, bab1f6e75b9a4aa8bf60faf966a23564; no voters:
I20260925 22:00:49.455917 14239 raft_consensus.cc:2781] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:49.455989 14239 raft_consensus.cc:482] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:49.456007 14239 raft_consensus.cc:3037] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:49.456794 14239 raft_consensus.cc:504] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.456912 14239 leader_election.cc:288] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [CANDIDATE]: Term 1 election: Requested vote from peers e149c6de27ba435b853eb40fff6f0bbe (127.9.112.66:38797), 4fdd541c08dd4897bd66ebb7b1a4d229 (127.9.112.67:39313)
I20260925 22:00:49.457119 14035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2764512c1b4f66916890434c65eeb6" candidate_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e149c6de27ba435b853eb40fff6f0bbe"
I20260925 22:00:49.457146 14168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2764512c1b4f66916890434c65eeb6" candidate_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229"
I20260925 22:00:49.457201 14035 raft_consensus.cc:3037] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:49.457226 14168 raft_consensus.cc:3037] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:49.457947 14035 raft_consensus.cc:2454] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab1f6e75b9a4aa8bf60faf966a23564 in term 1.
I20260925 22:00:49.457966 14168 raft_consensus.cc:2454] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab1f6e75b9a4aa8bf60faf966a23564 in term 1.
I20260925 22:00:49.458122 13835 leader_election.cc:302] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fdd541c08dd4897bd66ebb7b1a4d229, bab1f6e75b9a4aa8bf60faf966a23564; no voters:
I20260925 22:00:49.458205 14239 raft_consensus.cc:2781] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:49.458380 14239 raft_consensus.cc:686] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [term 1 LEADER]: Becoming Leader. State: Replica: bab1f6e75b9a4aa8bf60faf966a23564, State: Running, Role: LEADER
I20260925 22:00:49.458469 14239 consensus_queue.cc:237] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } }
I20260925 22:00:49.459407 13758 catalog_manager.cc:5495] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 reported cstate change: term changed from 0 to 1, leader changed from <none> to bab1f6e75b9a4aa8bf60faf966a23564 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab1f6e75b9a4aa8bf60faf966a23564" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 38735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:49.468101 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.465790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.9.112.65:38735)
0925 22:00:49.465862 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.465863 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.465871 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:49.465905 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.465905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.465922 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.465980 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:49.466111 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.466611 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.467365 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:49.467548 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:49.467602 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:49.467761 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:49.467871 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:49.467943 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.467978 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.467999 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:00:49.469940 14035 raft_consensus.cc:1261] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 1 FOLLOWER]: Refusing update from remote peer bab1f6e75b9a4aa8bf60faf966a23564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:49.469936 14168 raft_consensus.cc:1261] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 1 FOLLOWER]: Refusing update from remote peer bab1f6e75b9a4aa8bf60faf966a23564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:49.470299 14239 consensus_queue.cc:1035] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e149c6de27ba435b853eb40fff6f0bbe" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 38797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:49.470455 14241 consensus_queue.cc:1035] T 6b2764512c1b4f66916890434c65eeb6 P bab1f6e75b9a4aa8bf60faf966a23564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fdd541c08dd4897bd66ebb7b1a4d229" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:49.473730 14246 mvcc.cc:205] Tried to move back new op lower bound from 7333370468226322432 to 7333370468182605824. Current Snapshot: MvccSnapshot[applied={T|T < 7333370468226322432}]
I20260925 22:00:49.475732 13757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:49.475836 13757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:49.476540 13757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40e449fff1c044b282a4d7ce4fdd357d: Sending DeleteTablet for 3 replicas of tablet 6b2764512c1b4f66916890434c65eeb6
I20260925 22:00:49.476877 14148 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2764512c1b4f66916890434c65eeb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:49 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:34564
I20260925 22:00:49.476869 14015 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2764512c1b4f66916890434c65eeb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:49 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:58040
I20260925 22:00:49.477160 13882 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2764512c1b4f66916890434c65eeb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:49 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:35738
I20260925 22:00:49.477322 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13820
I20260925 22:00:49.477382 14255 tablet_replica.cc:330] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: stopping tablet replica
I20260925 22:00:49.478209 14255 raft_consensus.cc:2229] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:49.478332 14255 raft_consensus.cc:2258] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:49.478531 14254 tablet_replica.cc:330] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: stopping tablet replica
I20260925 22:00:49.478658 14254 raft_consensus.cc:2229] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:49.478767 14254 raft_consensus.cc:2258] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:49.479110 14255 ts_tablet_manager.cc:1905] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:49.479199 14254 ts_tablet_manager.cc:1905] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:49.480722 14255 ts_tablet_manager.cc:1918] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:49.480791 14255 log.cc:1196] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/6b2764512c1b4f66916890434c65eeb6
I20260925 22:00:49.480793 14254 ts_tablet_manager.cc:1918] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:49.480854 14254 log.cc:1196] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/6b2764512c1b4f66916890434c65eeb6
I20260925 22:00:49.481040 14255 ts_tablet_manager.cc:1939] T 6b2764512c1b4f66916890434c65eeb6 P e149c6de27ba435b853eb40fff6f0bbe: Deleting consensus metadata
I20260925 22:00:49.481106 14254 ts_tablet_manager.cc:1939] T 6b2764512c1b4f66916890434c65eeb6 P 4fdd541c08dd4897bd66ebb7b1a4d229: Deleting consensus metadata
I20260925 22:00:49.481482 13742 catalog_manager.cc:4841] TS e149c6de27ba435b853eb40fff6f0bbe (127.9.112.66:38797): tablet 6b2764512c1b4f66916890434c65eeb6 (table test-table [id=9ca65e5d36834b3eba690db7ebd5f19e]) successfully deleted
I20260925 22:00:49.481595 13742 catalog_manager.cc:4841] TS 4fdd541c08dd4897bd66ebb7b1a4d229 (127.9.112.67:39313): tablet 6b2764512c1b4f66916890434c65eeb6 (table test-table [id=9ca65e5d36834b3eba690db7ebd5f19e]) successfully deleted
W20260925 22:00:49.483122 13744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:38735 (error 108)
I20260925 22:00:49.483614 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.483513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:38735 (local address 127.0.0.1:35756)
0925 22:00:49.483552 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.483581 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 22:00:49.483727 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13953
W20260925 22:00:49.483731 13744 catalog_manager.cc:4584] TS bab1f6e75b9a4aa8bf60faf966a23564 (127.9.112.65:38735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b2764512c1b4f66916890434c65eeb6: Network error: Client connection negotiation failed: client connection to 127.9.112.65:38735: connect: Connection refused (error 111)
I20260925 22:00:49.488199 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14086
I20260925 22:00:49.492058 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 13727
2026-09-25T22:00:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1854 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T22:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:49Z Disabled control of system clock
I20260925 22:00:49.517796 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35437
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:45323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:00:49.599534 14261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.599772 14261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.599793 14261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.601059 14261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:49.601106 14261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.601118 14261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.601130 14261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:49.601142 14261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:49.602613 14261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.602852 14261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.603102 14261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:49.608026 14266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.608078 14267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.608232 14269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.608192 14261 server_base.cc:1034] running on GCE node
I20260925 22:00:49.608506 14261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:49.608762 14261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:49.609929 14261 hybrid_clock.cc:648] HybridClock initialized: now 1790373649609908 us; error 32 us; skew 500 ppm
I20260925 22:00:49.611419 14261 webserver.cc:460] Webserver started at http://127.9.112.126:37677/ using document root <none> and password file <none>
I20260925 22:00:49.611660 14261 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:49.611711 14261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:49.611831 14261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:49.612994 14261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "9135627f99134c7f8cb40e7bc1595848"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "a56226658772eba30edde5b575066691"
server_key_iv: "21d7261ec377d5ad1b082f1acd7ea97e"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.613355 14261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "9135627f99134c7f8cb40e7bc1595848"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "a56226658772eba30edde5b575066691"
server_key_iv: "21d7261ec377d5ad1b082f1acd7ea97e"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.614643 14261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:49.615483 14275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.615767 14261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:49.615875 14261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "9135627f99134c7f8cb40e7bc1595848"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "a56226658772eba30edde5b575066691"
server_key_iv: "21d7261ec377d5ad1b082f1acd7ea97e"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.615954 14261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:49.619460 14261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:49.619738 14261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:49.619855 14261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:49.623564 14261 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35437
I20260925 22:00:49.623615 14327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35437 every 8 connection(s)
I20260925 22:00:49.623927 14261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:49.624636 14328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:49.627406 14328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848: Bootstrap starting.
I20260925 22:00:49.627991 14328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:49.628437 14328 log.cc:824] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:49.629469 14328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848: No bootstrap required, opened a new log
I20260925 22:00:49.631112 14328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9135627f99134c7f8cb40e7bc1595848" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35437 } }
I20260925 22:00:49.631275 14328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:49.631307 14328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9135627f99134c7f8cb40e7bc1595848, State: Initialized, Role: FOLLOWER
I20260925 22:00:49.631449 14328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9135627f99134c7f8cb40e7bc1595848" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35437 } }
I20260925 22:00:49.631512 14328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:49.631543 14328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:49.631590 14328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:49.632057 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14261
I20260925 22:00:49.632148 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:49.632200 14328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9135627f99134c7f8cb40e7bc1595848" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35437 } }
I20260925 22:00:49.632332 9665 external_mini_cluster.cc:1428] Setting key 8f480c4fad58c18924f7cf9f5f2c4cbb
I20260925 22:00:49.632318 14328 leader_election.cc:302] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9135627f99134c7f8cb40e7bc1595848; no voters:
I20260925 22:00:49.632539 14328 leader_election.cc:288] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:49.632587 14331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:49.632707 14331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [term 1 LEADER]: Becoming Leader. State: Replica: 9135627f99134c7f8cb40e7bc1595848, State: Running, Role: LEADER
I20260925 22:00:49.632797 14331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9135627f99134c7f8cb40e7bc1595848" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35437 } }
I20260925 22:00:49.632841 14328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:49.633358 14332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9135627f99134c7f8cb40e7bc1595848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9135627f99134c7f8cb40e7bc1595848" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35437 } } }
I20260925 22:00:49.633450 14332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:49.633641 14342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:49.633618 14333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9135627f99134c7f8cb40e7bc1595848. Latest consensus state: current_term: 1 leader_uuid: "9135627f99134c7f8cb40e7bc1595848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9135627f99134c7f8cb40e7bc1595848" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35437 } } }
I20260925 22:00:49.633749 14333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:49.634174 14342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:49.635577 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.633014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.9.112.126:35437)
0925 22:00:49.633887 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.633898 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.633936 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:00:49.634069 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.634078 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.634127 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.634544 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.635083 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.635086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.635087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:49.635090 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.635129 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:49.635145 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.635226 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.635337 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.635406 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":78,"threads_started":1}
I20260925 22:00:49.636780 14342 catalog_manager.cc:1347] Generated new cluster ID: e31117734a1c4ff8a11f8a379f6bc393
I20260925 22:00:49.636826 14342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:49.654953 14342 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:49.655130 14342 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:49.662608 14342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848: Generated new TSK 0
I20260925 22:00:49.662935 14342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:00:49.669199 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35437
--builtin_ntp_servers=127.9.112.84:45323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:00:49.757951 14352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.758178 14352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.758203 14352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.759454 14352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.759502 14352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.759529 14352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:49.760972 14352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.761214 14352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.761525 14352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:49.767623 14358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.767629 14360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.767686 14357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.767916 14352 server_base.cc:1034] running on GCE node
I20260925 22:00:49.768134 14352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:49.768450 14352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:49.769613 14352 hybrid_clock.cc:648] HybridClock initialized: now 1790373649769599 us; error 33 us; skew 500 ppm
I20260925 22:00:49.770903 14352 webserver.cc:460] Webserver started at http://127.9.112.65:43097/ using document root <none> and password file <none>
I20260925 22:00:49.771140 14352 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:49.771226 14352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:49.771363 14352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:49.772465 14352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "5c43ebd7d55d447680e289f3b3d1ae22"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "5af18957bf06086ccd0cbe047fe9fa27"
server_key_iv: "6bcd58cd8b4a39a4f9c318bdccbbf79b"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.772794 14352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "5c43ebd7d55d447680e289f3b3d1ae22"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "5af18957bf06086ccd0cbe047fe9fa27"
server_key_iv: "6bcd58cd8b4a39a4f9c318bdccbbf79b"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.774063 14352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:49.774906 14366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.775063 14352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:49.775147 14352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "5c43ebd7d55d447680e289f3b3d1ae22"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "5af18957bf06086ccd0cbe047fe9fa27"
server_key_iv: "6bcd58cd8b4a39a4f9c318bdccbbf79b"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.775238 14352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:49.778570 14352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:49.778846 14352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:49.778967 14352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:49.779170 14352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:49.779462 14352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:49.779495 14352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.779520 14352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:49.779537 14352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.785135 14352 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:37053
I20260925 22:00:49.785207 14479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:37053 every 8 connection(s)
I20260925 22:00:49.785529 14352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:49.788606 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.786300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:54303 (local address 127.9.112.126:35437)
0925 22:00:49.786354 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.786357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.786579 (+ 222us) server_negotiation.cc:407] Connection header received
0925 22:00:49.787055 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.787056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.787064 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.787111 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.788325 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.788326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.788326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:49.788331 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.788382 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:49.788387 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.788457 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.788513 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.788530 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:00:49.788827 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.786286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35437 (local address 127.9.112.65:54303)
0925 22:00:49.786459 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.786501 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.786961 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.787193 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.787196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.787205 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:49.788146 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:49.788147 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:49.788178 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:49.788213 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:49.788435 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:49.788440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:49.788443 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.788520 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.788586 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260925 22:00:49.789348 14480 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35437
I20260925 22:00:49.789428 14480 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:49.789561 14480 heartbeater.cc:511] Master 127.9.112.126:35437 requested a full tablet report, sending...
I20260925 22:00:49.790074 14292 ts_manager.cc:194] Registered new tserver with Master: 5c43ebd7d55d447680e289f3b3d1ae22 (127.9.112.65:37053)
I20260925 22:00:49.795347 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14352
I20260925 22:00:49.795441 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:49.795562 9665 external_mini_cluster.cc:1428] Setting key 70dba37d952c2246e726942e55c3d00d
I20260925 22:00:49.796587 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35437
--builtin_ntp_servers=127.9.112.84:45323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:00:49.877455 14483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.877753 14483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.877779 14483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.879112 14483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.879168 14483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.879197 14483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:49.880736 14483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.880970 14483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.881237 14483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:49.886772 14491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.886785 14489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:49.886817 14488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:49.886806 14483 server_base.cc:1034] running on GCE node
I20260925 22:00:49.887276 14483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:49.887540 14483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:49.888701 14483 hybrid_clock.cc:648] HybridClock initialized: now 1790373649888662 us; error 58 us; skew 500 ppm
I20260925 22:00:49.890292 14483 webserver.cc:460] Webserver started at http://127.9.112.66:33443/ using document root <none> and password file <none>
I20260925 22:00:49.890604 14483 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:49.890686 14483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:49.890839 14483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:49.892309 14483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "65c65ca4747a467fbd2f34fd2c7f35e0"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "0198364e2468d989af388322b19a0749"
server_key_iv: "c534da4819bbb7df1cde2cadb27776fa"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.893123 14483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "65c65ca4747a467fbd2f34fd2c7f35e0"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "0198364e2468d989af388322b19a0749"
server_key_iv: "c534da4819bbb7df1cde2cadb27776fa"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.894977 14483 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:49.895797 14497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.895990 14483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:49.896082 14483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "65c65ca4747a467fbd2f34fd2c7f35e0"
format_stamp: "Formatted at 2026-09-25 22:00:49 on dist-test-slave-794d"
server_key: "0198364e2468d989af388322b19a0749"
server_key_iv: "c534da4819bbb7df1cde2cadb27776fa"
server_key_version: "encryptionkey@0"
I20260925 22:00:49.896157 14483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:49.899626 14483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:49.899931 14483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:49.900056 14483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:49.900274 14483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:49.900612 14483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:49.900645 14483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.900677 14483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:49.900700 14483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:49.906651 14483 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:35263
I20260925 22:00:49.906801 14610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:35263 every 8 connection(s)
I20260925 22:00:49.907104 14483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:49.910395 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.907908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:37981 (local address 127.9.112.126:35437)
0925 22:00:49.908001 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:49.908003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:49.908218 (+ 215us) server_negotiation.cc:407] Connection header received
0925 22:00:49.908794 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:49.908796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:49.908803 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:49.908860 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:49.910136 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:49.910137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:49.910138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:49.910143 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:49.910162 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:49.910172 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:49.910223 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:49.910303 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 22:00:49.910319 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 22:00:49.910593 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:49.907849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35437 (local address 127.9.112.66:37981)
0925 22:00:49.908069 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:49.908105 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:49.908687 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:49.908938 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:49.908943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:49.908953 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:49.909963 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:49.909963 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:49.909994 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:49.910025 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:49.910228 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:49.910234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:49.910237 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:49.910309 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 22:00:49.910377 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260925 22:00:49.911090 14611 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35437
I20260925 22:00:49.911177 14611 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:49.911194 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14483
I20260925 22:00:49.911310 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:49.911336 14611 heartbeater.cc:511] Master 127.9.112.126:35437 requested a full tablet report, sending...
I20260925 22:00:49.911476 9665 external_mini_cluster.cc:1428] Setting key 2bb21c640e42f3a38512a9089bb02d63
I20260925 22:00:49.911725 14292 ts_manager.cc:194] Registered new tserver with Master: 65c65ca4747a467fbd2f34fd2c7f35e0 (127.9.112.66:35263)
I20260925 22:00:49.912503 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35437
--builtin_ntp_servers=127.9.112.84:45323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 22:00:49.993742 14614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:49.993970 14614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:49.994004 14614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:49.995255 14614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:49.995312 14614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:49.995347 14614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:49.996831 14614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:49.997082 14614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:49.997380 14614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:50.002616 14619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.002616 14620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.002647 14622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:50.002962 14614 server_base.cc:1034] running on GCE node
I20260925 22:00:50.003165 14614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:50.003392 14614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:50.004559 14614 hybrid_clock.cc:648] HybridClock initialized: now 1790373650004543 us; error 32 us; skew 500 ppm
I20260925 22:00:50.005921 14614 webserver.cc:460] Webserver started at http://127.9.112.67:33403/ using document root <none> and password file <none>
I20260925 22:00:50.006142 14614 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:50.006192 14614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:50.006318 14614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:50.007408 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "34fa7d4c67c640d3a3ded94f88e932e5"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "6f8b37a17f4f39469133d2dfdb3abf6e"
server_key_iv: "db34b2f19c032f4506bca89cdcad8f0f"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.007944 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "34fa7d4c67c640d3a3ded94f88e932e5"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "6f8b37a17f4f39469133d2dfdb3abf6e"
server_key_iv: "db34b2f19c032f4506bca89cdcad8f0f"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.009366 14614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:50.010308 14628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.010512 14614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:50.010596 14614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "34fa7d4c67c640d3a3ded94f88e932e5"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "6f8b37a17f4f39469133d2dfdb3abf6e"
server_key_iv: "db34b2f19c032f4506bca89cdcad8f0f"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.010677 14614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:50.014700 14614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:50.014985 14614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:50.015101 14614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:50.015322 14614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:50.015635 14614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:50.015668 14614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.015690 14614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:50.015714 14614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.021592 14614 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:37085
I20260925 22:00:50.021651 14741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:37085 every 8 connection(s)
I20260925 22:00:50.022192 14614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:50.025285 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.022973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:52221 (local address 127.9.112.126:35437)
0925 22:00:50.023050 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.023052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.023251 (+ 199us) server_negotiation.cc:407] Connection header received
0925 22:00:50.023729 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.023730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.023736 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.023805 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.025064 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.025066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.025066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.025069 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.025084 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.025088 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.025144 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.025214 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.025226 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:00:50.025449 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.022922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35437 (local address 127.9.112.67:52221)
0925 22:00:50.023104 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.023140 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.023616 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.023882 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.023885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.023894 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:50.024907 (+ 1013us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:50.024908 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:50.024939 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:50.024970 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:50.025152 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:50.025155 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:50.025158 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:50.025210 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:00:50.025266 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260925 22:00:50.025904 14742 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35437
I20260925 22:00:50.026001 14742 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:50.026150 14742 heartbeater.cc:511] Master 127.9.112.126:35437 requested a full tablet report, sending...
I20260925 22:00:50.026572 14291 ts_manager.cc:194] Registered new tserver with Master: 34fa7d4c67c640d3a3ded94f88e932e5 (127.9.112.67:37085)
I20260925 22:00:50.027098 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14614
I20260925 22:00:50.027172 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:50.027290 9665 external_mini_cluster.cc:1428] Setting key 45a11d8b5565136cbb19f8f5f1109544
I20260925 22:00:50.028015 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:50.030159 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.029229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.9.112.126:35437)
0925 22:00:50.029291 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.029293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.029387 (+ 94us) server_negotiation.cc:407] Connection header received
0925 22:00:50.029431 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.029432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.029436 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.029480 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.029984 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.029985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.029986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.029989 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.030002 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.030006 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.030047 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.030078 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.030089 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:00:50.031735 14291 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:50.033484 14291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:50.037678 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.035856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:35263 (local address 127.0.0.1:59802)
0925 22:00:50.036192 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.036203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.036263 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.036743 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.036744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.036746 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:50.037233 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:50.037233 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:50.037244 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:50.037272 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:50.037511 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:50.037518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:50.037521 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:00:50.037586 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:00:50.037599 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260925 22:00:50.037699 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.035970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.9.112.66:35263)
0925 22:00:50.036161 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.036166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.036316 (+ 150us) server_negotiation.cc:407] Connection header received
0925 22:00:50.036366 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.036369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.036380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.036451 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.037386 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.037394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.037395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.037399 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.037431 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.037458 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.037516 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.037599 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.037614 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260925 22:00:50.037815 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.036292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.9.112.65:37053)
0925 22:00:50.036484 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.036488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.036497 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:50.036522 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.036524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.036532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.036599 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.037525 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.037533 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.037533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.037537 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.037572 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.037597 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.037652 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.037718 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.037734 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":125,"threads_started":1}
I20260925 22:00:50.037791 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.036059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37053 (local address 127.0.0.1:46652)
0925 22:00:50.036293 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.036301 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.036361 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.036604 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.036611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.036618 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:50.037386 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:50.037387 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:50.037394 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:50.037407 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:50.037658 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:50.037659 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:50.037660 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:00:50.037711 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 22:00:50.037724 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":67,"threads_started":1}
I20260925 22:00:50.038251 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.035970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.9.112.67:37085)
0925 22:00:50.036233 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.036236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.036245 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:50.036311 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.036315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.036330 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.036429 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.037373 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.037382 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.037382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.037387 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.037431 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.037458 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.037515 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.038170 (+ 655us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.038184 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":92,"threads_started":1}
I20260925 22:00:50.038192 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.035856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:37085 (local address 127.0.0.1:52246)
0925 22:00:50.036110 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.036153 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.036210 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.036744 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.036745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.036747 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:50.037225 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:50.037225 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:50.037244 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:50.037270 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:50.038096 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:50.038097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:50.038097 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 22:00:50.038141 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:00:50.038151 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":90,"threads_started":1}
I20260925 22:00:50.038897 14414 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd (DEFAULT_TABLE table=test-table [id=b6ed3b33e18e427ca7288e8aaaa0eb4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:50.038918 14545 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd (DEFAULT_TABLE table=test-table [id=b6ed3b33e18e427ca7288e8aaaa0eb4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:50.039206 14545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.039213 14414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.040776 14673 tablet_service.cc:1467] Processing CreateTablet for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd (DEFAULT_TABLE table=test-table [id=b6ed3b33e18e427ca7288e8aaaa0eb4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:50.041181 14673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.041944 14756 tablet_bootstrap.cc:492] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22: Bootstrap starting.
I20260925 22:00:50.042565 14756 tablet_bootstrap.cc:654] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.042838 14756 log.cc:824] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.043131 14757 tablet_bootstrap.cc:492] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Bootstrap starting.
I20260925 22:00:50.043515 14756 tablet_bootstrap.cc:492] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22: No bootstrap required, opened a new log
I20260925 22:00:50.043581 14756 ts_tablet_manager.cc:1397] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:50.043759 14757 tablet_bootstrap.cc:654] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.044234 14757 log.cc:824] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.044597 14759 tablet_bootstrap.cc:492] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Bootstrap starting.
I20260925 22:00:50.044991 14757 tablet_bootstrap.cc:492] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: No bootstrap required, opened a new log
I20260925 22:00:50.045058 14757 ts_tablet_manager.cc:1397] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:50.046061 14756 raft_consensus.cc:348] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.046248 14756 raft_consensus.cc:374] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.046293 14756 raft_consensus.cc:729] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c43ebd7d55d447680e289f3b3d1ae22, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.046416 14756 consensus_queue.cc:260] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.046676 14756 ts_tablet_manager.cc:1428] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:50.046706 14480 heartbeater.cc:503] Master 127.9.112.126:35437 was elected leader, sending a full tablet report...
I20260925 22:00:50.047489 14757 raft_consensus.cc:348] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.047650 14757 raft_consensus.cc:374] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.047683 14757 raft_consensus.cc:729] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c65ca4747a467fbd2f34fd2c7f35e0, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.047811 14757 consensus_queue.cc:260] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.048059 14757 ts_tablet_manager.cc:1428] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:50.048111 14611 heartbeater.cc:503] Master 127.9.112.126:35437 was elected leader, sending a full tablet report...
I20260925 22:00:50.048792 14759 tablet_bootstrap.cc:654] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.049059 14759 log.cc:824] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.049690 14759 tablet_bootstrap.cc:492] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: No bootstrap required, opened a new log
I20260925 22:00:50.049819 14759 ts_tablet_manager.cc:1397] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 22:00:50.051749 14759 raft_consensus.cc:348] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.051889 14759 raft_consensus.cc:374] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.051919 14759 raft_consensus.cc:729] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fa7d4c67c640d3a3ded94f88e932e5, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.052037 14759 consensus_queue.cc:260] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.052277 14759 ts_tablet_manager.cc:1428] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:00:50.052327 14742 heartbeater.cc:503] Master 127.9.112.126:35437 was elected leader, sending a full tablet report...
I20260925 22:00:50.064460 14761 raft_consensus.cc:482] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:50.064581 14761 raft_consensus.cc:504] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.064904 14761 leader_election.cc:288] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fa7d4c67c640d3a3ded94f88e932e5 (127.9.112.67:37085), 65c65ca4747a467fbd2f34fd2c7f35e0 (127.9.112.66:35263)
I20260925 22:00:50.066159 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.065116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:37085 (local address 127.9.112.65:40555)
0925 22:00:50.065176 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.065187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.065226 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.065384 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.065385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.065388 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:50.065847 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:50.065848 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:50.065854 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:50.065866 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:50.066025 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:50.066026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:50.066027 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:00:50.066075 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 22:00:50.066089 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 22:00:50.066159 14752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.065186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40555 (local address 127.9.112.67:37085)
0925 22:00:50.065244 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.065246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.065255 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:50.065287 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.065288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.065293 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.065374 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.065947 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.065948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.065948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.065953 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.065969 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.065974 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.066023 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.066077 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.066089 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:00:50.066262 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.065117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:35263 (local address 127.9.112.65:36587)
0925 22:00:50.065315 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.065324 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.065375 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.065495 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.065496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.065507 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:50.065971 (+ 464us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:00:50.065971 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:50.065978 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:00:50.065990 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:50.066145 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:50.066145 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:50.066146 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 22:00:50.066174 (+ 28us) client_negotiation.cc:240] Negotiation successful
0925 22:00:50.066187 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":21,"thread_start_us":130,"threads_started":1}
I20260925 22:00:50.066280 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.065189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:36587 (local address 127.9.112.66:35263)
0925 22:00:50.065244 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.065246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.065374 (+ 128us) server_negotiation.cc:407] Connection header received
0925 22:00:50.065427 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.065428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.065434 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.065489 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.066069 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.066071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.066071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.066075 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.066091 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.066095 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.066195 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.066210 (+ 15us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.066222 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:00:50.066502 14696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1e75b068a4e00ab5f2c1fe9dfc7cd" candidate_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" is_pre_election: true
I20260925 22:00:50.066603 14565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1e75b068a4e00ab5f2c1fe9dfc7cd" candidate_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" is_pre_election: true
I20260925 22:00:50.066661 14696 raft_consensus.cc:2454] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c43ebd7d55d447680e289f3b3d1ae22 in term 0.
I20260925 22:00:50.066748 14565 raft_consensus.cc:2454] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c43ebd7d55d447680e289f3b3d1ae22 in term 0.
I20260925 22:00:50.066908 14370 leader_election.cc:302] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fa7d4c67c640d3a3ded94f88e932e5, 5c43ebd7d55d447680e289f3b3d1ae22; no voters:
I20260925 22:00:50.067040 14761 raft_consensus.cc:2781] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:50.067114 14761 raft_consensus.cc:482] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:50.067137 14761 raft_consensus.cc:3037] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.067840 14761 raft_consensus.cc:504] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.067967 14761 leader_election.cc:288] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [CANDIDATE]: Term 1 election: Requested vote from peers 34fa7d4c67c640d3a3ded94f88e932e5 (127.9.112.67:37085), 65c65ca4747a467fbd2f34fd2c7f35e0 (127.9.112.66:35263)
I20260925 22:00:50.068152 14565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1e75b068a4e00ab5f2c1fe9dfc7cd" candidate_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0"
I20260925 22:00:50.068184 14696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c1e75b068a4e00ab5f2c1fe9dfc7cd" candidate_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fa7d4c67c640d3a3ded94f88e932e5"
I20260925 22:00:50.068239 14565 raft_consensus.cc:3037] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.068248 14696 raft_consensus.cc:3037] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.069121 14696 raft_consensus.cc:2454] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c43ebd7d55d447680e289f3b3d1ae22 in term 1.
I20260925 22:00:50.069121 14565 raft_consensus.cc:2454] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c43ebd7d55d447680e289f3b3d1ae22 in term 1.
I20260925 22:00:50.069319 14367 leader_election.cc:302] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c43ebd7d55d447680e289f3b3d1ae22, 65c65ca4747a467fbd2f34fd2c7f35e0; no voters:
I20260925 22:00:50.069408 14761 raft_consensus.cc:2781] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:50.069576 14761 raft_consensus.cc:686] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [term 1 LEADER]: Becoming Leader. State: Replica: 5c43ebd7d55d447680e289f3b3d1ae22, State: Running, Role: LEADER
I20260925 22:00:50.069732 14761 consensus_queue.cc:237] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } }
I20260925 22:00:50.070571 14291 catalog_manager.cc:5495] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c43ebd7d55d447680e289f3b3d1ae22 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c43ebd7d55d447680e289f3b3d1ae22" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37053 } health_report { overall_health: HEALTHY } } }
I20260925 22:00:50.077601 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.076706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.9.112.65:37053)
0925 22:00:50.076749 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.076750 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.076759 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:50.076819 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.076820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.076831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.076875 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.077392 (+ 517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.077393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.077393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.077396 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.077409 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.077412 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.077455 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.077514 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.077527 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:00:50.079370 14565 raft_consensus.cc:1261] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 1 FOLLOWER]: Refusing update from remote peer 5c43ebd7d55d447680e289f3b3d1ae22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:50.079413 14696 raft_consensus.cc:1261] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 1 FOLLOWER]: Refusing update from remote peer 5c43ebd7d55d447680e289f3b3d1ae22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:50.079728 14761 consensus_queue.cc:1035] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c65ca4747a467fbd2f34fd2c7f35e0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:50.079864 14766 consensus_queue.cc:1035] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 5c43ebd7d55d447680e289f3b3d1ae22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fa7d4c67c640d3a3ded94f88e932e5" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 37085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:50.082103 14769 mvcc.cc:205] Tried to move back new op lower bound from 7333370470722527232 to 7333370470686330880. Current Snapshot: MvccSnapshot[applied={T|T < 7333370470722527232}]
I20260925 22:00:50.082448 14770 mvcc.cc:205] Tried to move back new op lower bound from 7333370470722527232 to 7333370470686330880. Current Snapshot: MvccSnapshot[applied={T|T < 7333370470722527232}]
I20260925 22:00:50.085345 14292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:50.085433 14292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:50.086164 14292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9135627f99134c7f8cb40e7bc1595848: Sending DeleteTablet for 3 replicas of tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd
I20260925 22:00:50.086468 14673 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:50 UTC) from {username='slave'} at 127.0.0.1:52246
I20260925 22:00:50.086529 14545 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:50 UTC) from {username='slave'} at 127.0.0.1:59802
I20260925 22:00:50.086553 14414 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:50 UTC) from {username='slave'} at 127.0.0.1:46652
I20260925 22:00:50.086719 14778 tablet_replica.cc:330] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: stopping tablet replica
I20260925 22:00:50.086822 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14352
I20260925 22:00:50.086833 14778 raft_consensus.cc:2229] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:50.086939 14778 raft_consensus.cc:2258] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:50.087771 14778 ts_tablet_manager.cc:1905] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:50.088323 14779 tablet_replica.cc:330] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: stopping tablet replica
I20260925 22:00:50.088469 14779 raft_consensus.cc:2229] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:50.088567 14779 raft_consensus.cc:2258] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:50.089039 14779 ts_tablet_manager.cc:1905] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:50.089345 14778 ts_tablet_manager.cc:1918] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:50.089421 14778 log.cc:1196] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/44c1e75b068a4e00ab5f2c1fe9dfc7cd
I20260925 22:00:50.089664 14778 ts_tablet_manager.cc:1939] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 34fa7d4c67c640d3a3ded94f88e932e5: Deleting consensus metadata
I20260925 22:00:50.090322 14279 catalog_manager.cc:4841] TS 34fa7d4c67c640d3a3ded94f88e932e5 (127.9.112.67:37085): tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd (table test-table [id=b6ed3b33e18e427ca7288e8aaaa0eb4c]) successfully deleted
I20260925 22:00:50.090641 14779 ts_tablet_manager.cc:1918] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:50.090700 14779 log.cc:1196] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/44c1e75b068a4e00ab5f2c1fe9dfc7cd
I20260925 22:00:50.090907 14779 ts_tablet_manager.cc:1939] T 44c1e75b068a4e00ab5f2c1fe9dfc7cd P 65c65ca4747a467fbd2f34fd2c7f35e0: Deleting consensus metadata
I20260925 22:00:50.091279 14276 catalog_manager.cc:4841] TS 65c65ca4747a467fbd2f34fd2c7f35e0 (127.9.112.66:35263): tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd (table test-table [id=b6ed3b33e18e427ca7288e8aaaa0eb4c]) successfully deleted
W20260925 22:00:50.092134 14276 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:37053 (error 108)
I20260925 22:00:50.092664 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.092542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37053 (local address 127.0.0.1:46674)
0925 22:00:50.092591 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.092632 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:37053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 22:00:50.092656 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14483
W20260925 22:00:50.093557 14276 catalog_manager.cc:4584] TS 5c43ebd7d55d447680e289f3b3d1ae22 (127.9.112.65:37053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44c1e75b068a4e00ab5f2c1fe9dfc7cd: Network error: Client connection negotiation failed: client connection to 127.9.112.65:37053: connect: Connection refused (error 111)
I20260925 22:00:50.097433 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14614
I20260925 22:00:50.101475 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14261
2026-09-25T22:00:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (609 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 22:00:50.120733 9665 mini_oidc.cc:152] Starting JWKS server
I20260925 22:00:50.121845 9665 webserver.cc:460] Webserver started at https://127.0.0.1:40751/ using document root <none> and password file <none>
I20260925 22:00:50.121927 9665 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 22:00:50.122081 9665 webserver.cc:460] Webserver started at http://127.0.0.1:40217/ using document root <none> and password file <none>
2026-09-25T22:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:50Z Disabled control of system clock
I20260925 22:00:50.129527 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38027
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:38027
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40751/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/testchainca.cert with env {}
W20260925 22:00:50.210834 14787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:50.211064 14787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:50.211087 14787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:50.212410 14787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:50.212457 14787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:50.212468 14787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:50.212481 14787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:50.212491 14787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:00:50.212502 14787 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:00:50.212517 14787 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40751/jwks.json
I20260925 22:00:50.214184 14787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:38027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40751/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:50.214445 14787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:50.214660 14787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:50.219698 14793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.219807 14795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.219738 14792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:50.219972 14787 server_base.cc:1034] running on GCE node
I20260925 22:00:50.220204 14787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:50.220502 14787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:50.221647 14787 hybrid_clock.cc:648] HybridClock initialized: now 1790373650221631 us; error 35 us; skew 500 ppm
I20260925 22:00:50.222923 14787 webserver.cc:460] Webserver started at http://127.9.112.126:42891/ using document root <none> and password file <none>
I20260925 22:00:50.223145 14787 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:50.223191 14787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:50.223299 14787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:50.224424 14787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "aa254697f59e4631988c0add72e677cc"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "75327aaf499f84f78b2a73e72f87ea19"
server_key_iv: "9545cf92b650d9747e0faebf9f7266f7"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.224758 14787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "aa254697f59e4631988c0add72e677cc"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "75327aaf499f84f78b2a73e72f87ea19"
server_key_iv: "9545cf92b650d9747e0faebf9f7266f7"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.226008 14787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:50.226805 14801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.226980 14787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:50.227065 14787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "aa254697f59e4631988c0add72e677cc"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "75327aaf499f84f78b2a73e72f87ea19"
server_key_iv: "9545cf92b650d9747e0faebf9f7266f7"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.227130 14787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:50.255374 14787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:50.255717 14787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:50.255869 14787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:50.259850 14853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:38027 every 8 connection(s)
I20260925 22:00:50.259896 14787 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:38027
I20260925 22:00:50.260553 14787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:50.261735 14854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.264422 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14787
I20260925 22:00:50.264530 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:50.264699 9665 external_mini_cluster.cc:1428] Setting key 5f18508563b5aedda10059cd05adc033
I20260925 22:00:50.264966 14854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc: Bootstrap starting.
I20260925 22:00:50.265642 14854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.265916 14854 log.cc:824] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.266566 14854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc: No bootstrap required, opened a new log
I20260925 22:00:50.268328 14854 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa254697f59e4631988c0add72e677cc" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38027 } }
I20260925 22:00:50.268536 14854 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.268570 14854 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa254697f59e4631988c0add72e677cc, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.268709 14854 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa254697f59e4631988c0add72e677cc" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38027 } }
I20260925 22:00:50.268787 14854 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:50.268821 14854 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:50.268872 14854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.269476 14854 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa254697f59e4631988c0add72e677cc" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38027 } }
I20260925 22:00:50.269582 14854 leader_election.cc:302] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa254697f59e4631988c0add72e677cc; no voters:
I20260925 22:00:50.269773 14854 leader_election.cc:288] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:50.269856 14859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:50.269992 14859 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [term 1 LEADER]: Becoming Leader. State: Replica: aa254697f59e4631988c0add72e677cc, State: Running, Role: LEADER
I20260925 22:00:50.270077 14854 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:50.270831 14859 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa254697f59e4631988c0add72e677cc" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38027 } }
I20260925 22:00:50.271353 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.265203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44394 (local address 127.9.112.126:38027)
0925 22:00:50.265442 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.265450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.265477 (+ 27us) server_negotiation.cc:407] Connection header received
0925 22:00:50.265969 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.265977 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.266067 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.266204 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.266411 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.267364 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.268433 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.268994 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.269131 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.270607 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.270612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.270613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.270810 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.270864 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.270881 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.270946 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.271063 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.271163 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260925 22:00:50.271733 14860 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa254697f59e4631988c0add72e677cc. Latest consensus state: current_term: 1 leader_uuid: "aa254697f59e4631988c0add72e677cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa254697f59e4631988c0add72e677cc" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38027 } } }
I20260925 22:00:50.271855 14860 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.272070 14861 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa254697f59e4631988c0add72e677cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa254697f59e4631988c0add72e677cc" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38027 } } }
I20260925 22:00:50.272143 14861 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.272092 14875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:50.272698 14875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:50.272928 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:00:50.274534 14875 catalog_manager.cc:1347] Generated new cluster ID: d9cd7b7561f84a3bba8b143ffd7901fc
I20260925 22:00:50.274590 14875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:50.288584 14875 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:50.289196 14875 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:50.295030 14875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa254697f59e4631988c0add72e677cc: Generated new TSK 0
I20260925 22:00:50.295238 14875 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 22:00:50.320878 14884 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 22:00:50.320981 14884 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 22:00:50.321197 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.304237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.9.112.126:38027)
0925 22:00:50.304289 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.304291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.304428 (+ 137us) server_negotiation.cc:407] Connection header received
0925 22:00:50.304500 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.304501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.304515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.304590 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:00:50.304735 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.305597 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.306803 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.307128 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.307209 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.307452 (+ 243us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:00:50.320962 (+ 13510us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 22:00:50.321117 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":29}
W20260925 22:00:50.321291 14857 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 22:00:50.321663 14883 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:50.304132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38027 (local address 127.0.0.1:44402)
0925 22:00:50.304306 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.304316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.304404 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.304602 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.304603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.304605 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:00:50.304669 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.304671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.305778 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.305780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.306668 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.306675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.307232 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.307233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.307296 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.307329 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:00:50.321193 (+ 13864us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 22:00:50.321348 (+ 155us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.112.126:38027: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.112.126:38027: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 22:00:50.322111 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14787
2026-09-25T22:00:50Z chronyd exiting
I20260925 22:00:50.358144 9665 test_util.cc:175] -----------------------------------------------
I20260925 22:00:50.358250 9665 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790373628768857-9665-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 22:00:50.359099 9665 mini_oidc.cc:152] Starting JWKS server
I20260925 22:00:50.359822 9665 webserver.cc:460] Webserver started at https://127.0.0.1:44015/ using document root <none> and password file <none>
I20260925 22:00:50.359867 9665 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 22:00:50.360004 9665 webserver.cc:460] Webserver started at http://127.0.0.1:46631/ using document root <none> and password file <none>
2026-09-25T22:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:50Z Disabled control of system clock
I20260925 22:00:50.367978 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40411
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40411
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44015/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/testchainca.cert with env {}
W20260925 22:00:50.455286 14890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:50.455523 14890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:50.455554 14890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:50.456854 14890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:50.456903 14890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:50.456919 14890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:50.456936 14890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:50.456950 14890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:00:50.456966 14890 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:00:50.456990 14890 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44015/jwks.json
I20260925 22:00:50.458578 14890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44015/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:50.458827 14890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:50.459023 14890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:50.463900 14895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.463917 14898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.463900 14896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:50.464407 14890 server_base.cc:1034] running on GCE node
I20260925 22:00:50.464692 14890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:50.465037 14890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:50.466188 14890 hybrid_clock.cc:648] HybridClock initialized: now 1790373650466154 us; error 50 us; skew 500 ppm
I20260925 22:00:50.467494 14890 webserver.cc:460] Webserver started at http://127.9.112.126:40721/ using document root <none> and password file <none>
I20260925 22:00:50.467713 14890 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:50.467763 14890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:50.467890 14890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:50.469009 14890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "172a4161cb804a5dada813b7179b14b1"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "4f7c4c20666ff7f61e88428a0a7d83aa"
server_key_iv: "493befee0a1285c2b78fea50b297bb69"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.469357 14890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "172a4161cb804a5dada813b7179b14b1"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "4f7c4c20666ff7f61e88428a0a7d83aa"
server_key_iv: "493befee0a1285c2b78fea50b297bb69"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.470674 14890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:50.471517 14904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.471797 14890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:50.471899 14890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "172a4161cb804a5dada813b7179b14b1"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "4f7c4c20666ff7f61e88428a0a7d83aa"
server_key_iv: "493befee0a1285c2b78fea50b297bb69"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.471977 14890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:50.479930 14890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:50.480283 14890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:50.480458 14890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:50.484586 14890 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40411
I20260925 22:00:50.484617 14956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40411 every 8 connection(s)
I20260925 22:00:50.484989 14890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:50.485998 14957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.488556 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1: Bootstrap starting.
I20260925 22:00:50.489180 14957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.489432 14957 log.cc:824] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.490170 14957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1: No bootstrap required, opened a new log
I20260925 22:00:50.491828 14957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172a4161cb804a5dada813b7179b14b1" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40411 } }
I20260925 22:00:50.491991 14957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.492015 14957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 172a4161cb804a5dada813b7179b14b1, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.492144 14957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172a4161cb804a5dada813b7179b14b1" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40411 } }
I20260925 22:00:50.492228 14957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:50.492264 14957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:50.492298 14957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.492882 14957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172a4161cb804a5dada813b7179b14b1" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40411 } }
I20260925 22:00:50.492971 14957 leader_election.cc:302] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 172a4161cb804a5dada813b7179b14b1; no voters:
I20260925 22:00:50.493145 14957 leader_election.cc:288] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:50.493191 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14890
I20260925 22:00:50.493196 14960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:50.493280 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:50.493353 14960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [term 1 LEADER]: Becoming Leader. State: Replica: 172a4161cb804a5dada813b7179b14b1, State: Running, Role: LEADER
I20260925 22:00:50.493417 14957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:50.493455 9665 external_mini_cluster.cc:1428] Setting key 6556660a4c45dddc34a268a02057a980
I20260925 22:00:50.493474 14960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172a4161cb804a5dada813b7179b14b1" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40411 } }
I20260925 22:00:50.493806 14960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 172a4161cb804a5dada813b7179b14b1. Latest consensus state: current_term: 1 leader_uuid: "172a4161cb804a5dada813b7179b14b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172a4161cb804a5dada813b7179b14b1" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40411 } } }
I20260925 22:00:50.493875 14960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.494068 14962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "172a4161cb804a5dada813b7179b14b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172a4161cb804a5dada813b7179b14b1" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40411 } } }
I20260925 22:00:50.494179 14962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.494318 14967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:50.494827 14967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:50.497429 14967 catalog_manager.cc:1347] Generated new cluster ID: d9ef33f81c684a2a9f23998811b98876
I20260925 22:00:50.497489 14967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:50.499619 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.493891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.9.112.126:40411)
0925 22:00:50.494312 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.494321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.494347 (+ 26us) server_negotiation.cc:407] Connection header received
0925 22:00:50.494465 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.494472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.494566 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.494961 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.495054 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.496540 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.497401 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.498032 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.498161 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.498882 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.498886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.498887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.499104 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.499151 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.499167 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.499251 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.499311 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.499398 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":74,"threads_started":1}
I20260925 22:00:50.501350 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:00:50.510221 14967 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:50.511008 14967 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:50.515882 14967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 172a4161cb804a5dada813b7179b14b1: Generated new TSK 0
I20260925 22:00:50.516103 14967 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 22:00:50.546990 14987 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 22:00:50.547084 14987 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 22:00:50.547384 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.525171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.9.112.126:40411)
0925 22:00:50.525220 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.525222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.525267 (+ 45us) server_negotiation.cc:407] Connection header received
0925 22:00:50.525357 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.525358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.525368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.525425 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:00:50.525558 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.526062 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.527252 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.527482 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.527543 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.527772 (+ 229us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:00:50.547126 (+ 19354us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 22:00:50.547318 (+ 192us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 22:00:50.547434 14968 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 22:00:50.547458 14986 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:50.525068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40411 (local address 127.0.0.1:60998)
0925 22:00:50.525199 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.525211 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.525287 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.525437 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.525438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.525440 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:00:50.525498 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.525501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.526128 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.526129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.527098 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.527103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.527557 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.527558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.527619 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.527627 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:00:50.547240 (+ 19613us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 22:00:50.547363 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.112.126:40411: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260925 22:00:50.547729 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14890
2026-09-25T22:00:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 22:00:50.584790 9665 mini_oidc.cc:152] Starting JWKS server
I20260925 22:00:50.585433 9665 webserver.cc:460] Webserver started at https://127.0.0.1:32903/ using document root <none> and password file <none>
I20260925 22:00:50.585479 9665 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 22:00:50.585608 9665 webserver.cc:460] Webserver started at http://127.0.0.1:45385/ using document root <none> and password file <none>
2026-09-25T22:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:50Z Disabled control of system clock
I20260925 22:00:50.593026 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35121
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:37263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35121
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32903/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 22:00:50.675029 14993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:50.675261 14993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:50.675284 14993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:50.676487 14993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:50.676537 14993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:50.676553 14993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:50.676569 14993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:50.676584 14993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:00:50.676599 14993 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:00:50.676621 14993 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32903/jwks.json
I20260925 22:00:50.678036 14993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32903/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:50.678251 14993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:50.678807 14993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:50.684491 14998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.684520 15001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.684520 14999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:50.684558 14993 server_base.cc:1034] running on GCE node
I20260925 22:00:50.685012 14993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:50.685355 14993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:50.686506 14993 hybrid_clock.cc:648] HybridClock initialized: now 1790373650686487 us; error 33 us; skew 500 ppm
I20260925 22:00:50.687800 14993 webserver.cc:460] Webserver started at http://127.9.112.126:35973/ using document root <none> and password file <none>
I20260925 22:00:50.688043 14993 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:50.688097 14993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:50.688230 14993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:50.689400 14993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "185c73d6d149435ab8da3d2bee9f6c37"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "f9dd45159c4d6ed3030c83f2f3907def"
server_key_iv: "e9f8f71d6f0ebb10bb0888eb5089f600"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.689752 14993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "185c73d6d149435ab8da3d2bee9f6c37"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "f9dd45159c4d6ed3030c83f2f3907def"
server_key_iv: "e9f8f71d6f0ebb10bb0888eb5089f600"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.691023 14993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:00:50.691811 15007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.691992 14993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:50.692080 14993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "185c73d6d149435ab8da3d2bee9f6c37"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "f9dd45159c4d6ed3030c83f2f3907def"
server_key_iv: "e9f8f71d6f0ebb10bb0888eb5089f600"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.692152 14993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:50.725493 14993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:50.725847 14993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:50.725986 14993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:50.730443 14993 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35121
I20260925 22:00:50.730489 15059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35121 every 8 connection(s)
I20260925 22:00:50.730937 14993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:50.731621 15060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.734387 15060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37: Bootstrap starting.
I20260925 22:00:50.734956 15060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.735200 15060 log.cc:824] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.735937 15060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37: No bootstrap required, opened a new log
I20260925 22:00:50.737731 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 14993
I20260925 22:00:50.737823 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:50.737682 15060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "185c73d6d149435ab8da3d2bee9f6c37" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35121 } }
I20260925 22:00:50.737857 15060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.737890 15060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185c73d6d149435ab8da3d2bee9f6c37, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.737995 9665 external_mini_cluster.cc:1428] Setting key d3f76f3fb66744f92926a9d8d9ba57c5
I20260925 22:00:50.738039 15060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "185c73d6d149435ab8da3d2bee9f6c37" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35121 } }
I20260925 22:00:50.738116 15060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:50.738183 15060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:50.738229 15060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.738945 15060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "185c73d6d149435ab8da3d2bee9f6c37" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35121 } }
I20260925 22:00:50.739055 15060 leader_election.cc:302] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 185c73d6d149435ab8da3d2bee9f6c37; no voters:
I20260925 22:00:50.739215 15060 leader_election.cc:288] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:50.739279 15065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:50.739436 15065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [term 1 LEADER]: Becoming Leader. State: Replica: 185c73d6d149435ab8da3d2bee9f6c37, State: Running, Role: LEADER
I20260925 22:00:50.739468 15060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:50.739535 15065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "185c73d6d149435ab8da3d2bee9f6c37" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35121 } }
I20260925 22:00:50.739961 15066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 185c73d6d149435ab8da3d2bee9f6c37. Latest consensus state: current_term: 1 leader_uuid: "185c73d6d149435ab8da3d2bee9f6c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "185c73d6d149435ab8da3d2bee9f6c37" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35121 } } }
I20260925 22:00:50.740051 15066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.740207 15067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "185c73d6d149435ab8da3d2bee9f6c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "185c73d6d149435ab8da3d2bee9f6c37" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35121 } } }
I20260925 22:00:50.740288 15067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.740624 15074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:50.741168 15074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:50.743220 15074 catalog_manager.cc:1347] Generated new cluster ID: 57a2a1e6667446fbb992472535613d99
I20260925 22:00:50.743278 15074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:50.744586 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.738428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41482 (local address 127.9.112.126:35121)
0925 22:00:50.738673 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.738683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.738720 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:00:50.738840 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.738848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.738931 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.739082 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.739417 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.741463 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.742374 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.742890 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.743006 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.743756 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.743760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.743760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.743965 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.744007 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.744022 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.744074 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.744276 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.744384 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
I20260925 22:00:50.745476 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:00:50.758415 15074 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:50.759156 15074 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:50.763114 15074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 185c73d6d149435ab8da3d2bee9f6c37: Generated new TSK 0
I20260925 22:00:50.763306 15074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:00:50.779333 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.775501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41488 (local address 127.9.112.126:35121)
0925 22:00:50.775547 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.775549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.775622 (+ 73us) server_negotiation.cc:407] Connection header received
0925 22:00:50.775683 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.775684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.775695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.775781 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:00:50.775888 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.776722 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.777576 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.777908 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.777967 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.778125 (+ 158us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:00:50.779118 (+ 993us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32903/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 22:00:50.779253 (+ 135us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32903/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 22:00:50.779402 15064 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32903/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 22:00:50.781232 15089 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:50.775410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35121 (local address 127.0.0.1:41488)
0925 22:00:50.775558 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.775567 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.775623 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.775781 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.775781 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.775783 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:00:50.775832 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.775834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.776853 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.776855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.777495 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.777498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.777982 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.777982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.778029 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.778034 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:00:50.779218 (+ 1184us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32903/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 22:00:50.781133 (+ 1915us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.112.126:35121: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32903/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":1732,"thread_start_us":91,"threads_started":1}
I20260925 22:00:50.781705 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 14993
E20260925 22:00:50.790164 15090 webserver.cc:533] Webserver: sslize failed:
E20260925 22:00:50.790244 15090 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T22:00:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (233 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T22:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:50Z Disabled control of system clock
I20260925 22:00:50.825309 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44013
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:44013
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/testchainca.cert with env {}
W20260925 22:00:50.909318 15094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:50.909543 15094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:50.909574 15094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:50.910794 15094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:50.910836 15094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:50.910853 15094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:50.910867 15094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:50.910883 15094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 22:00:50.910907 15094 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 22:00:50.910923 15094 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 22:00:50.912542 15094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:44013
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:44013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:50.912817 15094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:50.913028 15094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:50.918109 15099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.918100 15100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:50.918103 15102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:50.918674 15094 server_base.cc:1034] running on GCE node
I20260925 22:00:50.918943 15094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:50.919263 15094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:50.920419 15094 hybrid_clock.cc:648] HybridClock initialized: now 1790373650920403 us; error 31 us; skew 500 ppm
I20260925 22:00:50.921749 15094 webserver.cc:460] Webserver started at http://127.9.112.126:37303/ using document root <none> and password file <none>
I20260925 22:00:50.922014 15094 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:50.922065 15094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:50.922173 15094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:50.923211 15094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "360f37d7423346bb855da56c6459b877"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "1c2e054dc5b89852b0e6625fd616d261"
server_key_iv: "4edf69ed04828d0e91d2a8d925d2905c"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.923548 15094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "360f37d7423346bb855da56c6459b877"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "1c2e054dc5b89852b0e6625fd616d261"
server_key_iv: "4edf69ed04828d0e91d2a8d925d2905c"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.924926 15094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:50.925637 15108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:50.925778 15094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:50.925864 15094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "360f37d7423346bb855da56c6459b877"
format_stamp: "Formatted at 2026-09-25 22:00:50 on dist-test-slave-794d"
server_key: "1c2e054dc5b89852b0e6625fd616d261"
server_key_iv: "4edf69ed04828d0e91d2a8d925d2905c"
server_key_version: "encryptionkey@0"
I20260925 22:00:50.925930 15094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:50.939620 15094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:50.939988 15094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:50.940130 15094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:50.944170 15094 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:44013
I20260925 22:00:50.944205 15160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:44013 every 8 connection(s)
I20260925 22:00:50.944608 15094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:50.945634 15161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:50.947929 15161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877: Bootstrap starting.
I20260925 22:00:50.948532 15161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:50.948770 15161 log.cc:824] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:50.949467 15161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877: No bootstrap required, opened a new log
I20260925 22:00:50.949826 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15094
I20260925 22:00:50.949918 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:50.950085 9665 external_mini_cluster.cc:1428] Setting key 36042f67ef92b2789acc4875fc3cf84b
I20260925 22:00:50.951905 15161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360f37d7423346bb855da56c6459b877" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44013 } }
I20260925 22:00:50.952106 15161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:50.952142 15161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360f37d7423346bb855da56c6459b877, State: Initialized, Role: FOLLOWER
I20260925 22:00:50.952291 15161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360f37d7423346bb855da56c6459b877" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44013 } }
I20260925 22:00:50.952410 15161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:50.952452 15161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:50.952495 15161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:50.953334 15161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360f37d7423346bb855da56c6459b877" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44013 } }
I20260925 22:00:50.953470 15161 leader_election.cc:302] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 360f37d7423346bb855da56c6459b877; no voters:
I20260925 22:00:50.953629 15161 leader_election.cc:288] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:50.953716 15166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:50.953902 15161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:50.953995 15166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [term 1 LEADER]: Becoming Leader. State: Replica: 360f37d7423346bb855da56c6459b877, State: Running, Role: LEADER
I20260925 22:00:50.954149 15166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360f37d7423346bb855da56c6459b877" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44013 } }
I20260925 22:00:50.954628 15167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "360f37d7423346bb855da56c6459b877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360f37d7423346bb855da56c6459b877" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44013 } } }
I20260925 22:00:50.954725 15167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [sys.catalog]: This master's current role is: LEADER
W20260925 22:00:50.955768 15181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:00:50.955830 15181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:00:50.955899 15175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:50.956060 15175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:50.955734 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 360f37d7423346bb855da56c6459b877. Latest consensus state: current_term: 1 leader_uuid: "360f37d7423346bb855da56c6459b877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360f37d7423346bb855da56c6459b877" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 44013 } } }
I20260925 22:00:50.956414 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:50.958006 15175 catalog_manager.cc:1347] Generated new cluster ID: 5208facbbb0045ea8330e9ef0d400361
I20260925 22:00:50.958063 15175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:50.958084 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.950738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.9.112.126:44013)
0925 22:00:50.951035 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.951044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.951078 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:00:50.951853 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.951863 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.951950 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.952120 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:50.952489 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.954201 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.955600 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.956457 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.956700 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.957343 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:50.957347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:50.957347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:00:50.957573 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:50.957614 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:00:50.957631 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:50.957707 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:50.957795 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 22:00:50.957881 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":117,"threads_started":1}
I20260925 22:00:50.959045 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:00:50.963797 15175 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:50.964558 15175 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:50.967396 15175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 360f37d7423346bb855da56c6459b877: Generated new TSK 0
I20260925 22:00:50.967629 15175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:00:51.055176 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:50.976413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.9.112.126:44013)
0925 22:00:50.976462 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:50.976465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:50.976500 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:00:50.976576 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:50.976577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:50.976592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:50.976685 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0925 22:00:50.976796 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.977581 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.978484 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.978795 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.978865 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.979039 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 22:00:51.054884 (+ 75845us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 22:00:51.055077 (+ 193us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46392: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 22:00:51.055264 15165 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46392: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 22:00:51.055297 15190 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:50.976279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:44013 (local address 127.0.0.1:46392)
0925 22:00:50.976439 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:50.976449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:50.976509 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:50.976682 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:50.976682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:50.976684 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 22:00:50.976742 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.976744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.977767 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.977769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.978399 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:50.978402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:50.978880 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:50.978881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:50.978929 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:50.978935 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 22:00:51.055053 (+ 76118us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 22:00:51.055201 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.112.126:44013: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260925 22:00:51.055655 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15094
2026-09-25T22:00:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (254 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:51 dist-test-slave-794d krb5kdc[15202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:51 dist-test-slave-794d krb5kdc[15202](info): set up 2 sockets
Sep 25 22:00:51 dist-test-slave-794d krb5kdc[15202](info): commencing operation
krb5kdc: starting...
W20260925 22:00:52.188443 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:00:52.308254 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:41165
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:41165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:52.390060 15215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:52.390316 15215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:52.390340 15215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:52.391615 15215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:52.391669 15215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:52.391681 15215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:52.391698 15215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:52.391709 15215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:52.393532 15215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:41165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:41165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:52.393885 15215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:52.394208 15215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:52.399137 15220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.399180 15223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.399183 15221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:52.399447 15215 server_base.cc:1034] running on GCE node
I20260925 22:00:52.399679 15215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:52.399973 15215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:52.401134 15215 hybrid_clock.cc:648] HybridClock initialized: now 1790373652401112 us; error 32 us; skew 500 ppm
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:52.407773 15215 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:00:52.408236 15215 webserver.cc:460] Webserver started at http://127.9.112.126:43125/ using document root <none> and password file <none>
I20260925 22:00:52.408480 15215 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:52.408532 15215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:52.408654 15215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:52.409718 15215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "046531870e2b440ebed8f8b5eacc087d"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "c2eb7ac1175a02aebdc5eaa7e01b7feb"
server_key_iv: "46c7f9000fc221dafd1954adcae05fa3"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.410048 15215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "046531870e2b440ebed8f8b5eacc087d"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "c2eb7ac1175a02aebdc5eaa7e01b7feb"
server_key_iv: "46c7f9000fc221dafd1954adcae05fa3"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.411372 15215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:52.412194 15230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.412400 15215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:52.412494 15215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "046531870e2b440ebed8f8b5eacc087d"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "c2eb7ac1175a02aebdc5eaa7e01b7feb"
server_key_iv: "46c7f9000fc221dafd1954adcae05fa3"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.412567 15215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:52.439651 15215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:52.442387 15215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:52.442549 15215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:52.447393 15215 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:41165
I20260925 22:00:52.447422 15282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:41165 every 8 connection(s)
I20260925 22:00:52.447839 15215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:52.448799 15283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:52.451110 15283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d: Bootstrap starting.
I20260925 22:00:52.451835 15283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:52.452152 15283 log.cc:824] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:52.453426 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15215
I20260925 22:00:52.453512 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:52.453680 9665 external_mini_cluster.cc:1428] Setting key e8c150eb3d70288497efc08dca3155c1
I20260925 22:00:52.455106 15283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d: No bootstrap required, opened a new log
I20260925 22:00:52.457401 15283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046531870e2b440ebed8f8b5eacc087d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41165 } }
I20260925 22:00:52.457579 15283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:52.457607 15283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046531870e2b440ebed8f8b5eacc087d, State: Initialized, Role: FOLLOWER
I20260925 22:00:52.457743 15283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046531870e2b440ebed8f8b5eacc087d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41165 } }
I20260925 22:00:52.457808 15283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:52.457836 15283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:52.457873 15283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:52.458954 15283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046531870e2b440ebed8f8b5eacc087d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41165 } }
I20260925 22:00:52.459056 15283 leader_election.cc:302] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 046531870e2b440ebed8f8b5eacc087d; no voters:
I20260925 22:00:52.459235 15283 leader_election.cc:288] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:52.459285 15288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:52.459435 15288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [term 1 LEADER]: Becoming Leader. State: Replica: 046531870e2b440ebed8f8b5eacc087d, State: Running, Role: LEADER
I20260925 22:00:52.459476 15283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:52.459522 15288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046531870e2b440ebed8f8b5eacc087d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41165 } }
I20260925 22:00:52.459869 15290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "046531870e2b440ebed8f8b5eacc087d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046531870e2b440ebed8f8b5eacc087d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41165 } } }
I20260925 22:00:52.459861 15288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 046531870e2b440ebed8f8b5eacc087d. Latest consensus state: current_term: 1 leader_uuid: "046531870e2b440ebed8f8b5eacc087d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046531870e2b440ebed8f8b5eacc087d" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41165 } } }
I20260925 22:00:52.459949 15290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:52.459955 15288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:52.460285 15296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:52.461491 15296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:52.465585 15296 catalog_manager.cc:1347] Generated new cluster ID: d19f31ec0f364c23ae3e1da72c072e6e
I20260925 22:00:52.465687 15296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:52.480562 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.454388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.9.112.126:41165)
0925 22:00:52.459665 (+ 5277us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:52.459676 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:52.459721 (+ 45us) server_negotiation.cc:407] Connection header received
0925 22:00:52.459848 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:52.459857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:52.459922 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:52.460111 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:52.460877 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.462244 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.463362 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.464011 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.464137 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.475925 (+ 11788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:52.475933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:52.475933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:52.475950 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:52.478394 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.478891 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.478895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.478895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.479028 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.479260 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.479261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.479261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.479643 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:52.479789 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:52.480073 (+ 284us) server_negotiation.cc:299] Negotiation successful
0925 22:00:52.480247 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5220,"thread_start_us":82,"threads_started":1}
I20260925 22:00:52.498699 15296 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:52.499429 15296 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:52.506356 15296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 046531870e2b440ebed8f8b5eacc087d: Generated new TSK 0
I20260925 22:00:52.506577 15296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:00:52.539320 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41165
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:52.632995 15309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:52.633227 15309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:52.633255 15309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:52.634500 15309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:52.634548 15309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:52.634577 15309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:52.636235 15309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:52.636526 15309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:52.636765 15309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:52.641707 15314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.641762 15317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.641896 15315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:52.641992 15309 server_base.cc:1034] running on GCE node
I20260925 22:00:52.642189 15309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:52.642452 15309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:52.643635 15309 hybrid_clock.cc:648] HybridClock initialized: now 1790373652643616 us; error 36 us; skew 500 ppm
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:52.646346 15309 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:00:52.646883 15309 webserver.cc:460] Webserver started at http://127.9.112.65:34083/ using document root <none> and password file <none>
I20260925 22:00:52.647092 15309 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:52.647149 15309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:52.647274 15309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:52.648370 15309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "2ea7188d71b543119513a5398f0cc948"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "32507e0aa1dc423871b44a34326cf7eb"
server_key_iv: "278b0c48b67d696b0e7ce240988be478"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.648747 15309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "2ea7188d71b543119513a5398f0cc948"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "32507e0aa1dc423871b44a34326cf7eb"
server_key_iv: "278b0c48b67d696b0e7ce240988be478"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.650154 15309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:52.651010 15324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.651181 15309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:52.651273 15309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "2ea7188d71b543119513a5398f0cc948"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "32507e0aa1dc423871b44a34326cf7eb"
server_key_iv: "278b0c48b67d696b0e7ce240988be478"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.651345 15309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:52.659767 15309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:52.662009 15309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:52.662158 15309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:52.662384 15309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:52.662686 15309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:52.662719 15309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.662748 15309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:52.662771 15309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.668454 15309 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:36265
I20260925 22:00:52.668509 15437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:36265 every 8 connection(s)
I20260925 22:00:52.668828 15309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:52.675493 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15309
I20260925 22:00:52.675580 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:52.675769 9665 external_mini_cluster.cc:1428] Setting key 187a54208bf668125b9e601e1846ddc1
I20260925 22:00:52.679153 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.669644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:53165 (local address 127.9.112.126:41165)
0925 22:00:52.669731 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:52.669733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:52.669909 (+ 176us) server_negotiation.cc:407] Connection header received
0925 22:00:52.670908 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:52.670910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:52.670925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:52.670987 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:52.671634 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.672150 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.673303 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.673549 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.673620 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.676077 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:52.676078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:52.676079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:52.676090 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:52.677673 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.678078 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.678080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.678080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.678160 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.678450 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.678451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.678451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.678594 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:52.678657 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:52.678965 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 22:00:52.679054 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 22:00:52.679260 15440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.669568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:41165 (local address 127.9.112.65:53165)
0925 22:00:52.669799 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:52.669837 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:52.670775 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:52.671056 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:52.671060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:52.671329 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:52.671529 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:52.671533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.672263 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.672264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:52.673201 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:52.673204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.673658 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.673693 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:52.673823 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.674480 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:52.674490 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:52.675942 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:52.677764 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:52.677770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:52.677772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:52.678014 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:52.678248 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:52.678249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:52.678250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:52.678382 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:52.678674 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:52.678688 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:52.678826 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:00:52.678924 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 22:00:52.679062 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260925 22:00:52.679798 15438 heartbeater.cc:347] Connected to a master server at 127.9.112.126:41165
I20260925 22:00:52.679912 15438 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:52.680135 15438 heartbeater.cc:511] Master 127.9.112.126:41165 requested a full tablet report, sending...
I20260925 22:00:52.680742 15247 ts_manager.cc:194] Registered new tserver with Master: 2ea7188d71b543119513a5398f0cc948 (127.9.112.65:36265)
I20260925 22:00:52.681344 15247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:53165
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:00:52.703723 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41165
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:52.784705 15443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:52.784950 15443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:52.784976 15443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:52.786315 15443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:52.786383 15443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:52.786413 15443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:52.788226 15443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:52.788489 15443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:52.788694 15443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:52.793793 15449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.793716 15451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.793716 15448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:52.794004 15443 server_base.cc:1034] running on GCE node
I20260925 22:00:52.794196 15443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:52.794410 15443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:52.795568 15443 hybrid_clock.cc:648] HybridClock initialized: now 1790373652795549 us; error 31 us; skew 500 ppm
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:52.798573 15443 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:00:52.799127 15443 webserver.cc:460] Webserver started at http://127.9.112.66:43357/ using document root <none> and password file <none>
I20260925 22:00:52.799356 15443 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:52.799408 15443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:52.799540 15443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:52.800702 15443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "a6ecf946296d46b59e9d5f7ac179334c"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "b5d5657321edb6fdf1c8800d798ec653"
server_key_iv: "3739dcf25aff7f3c78ff29f599e1a273"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.801036 15443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "a6ecf946296d46b59e9d5f7ac179334c"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "b5d5657321edb6fdf1c8800d798ec653"
server_key_iv: "3739dcf25aff7f3c78ff29f599e1a273"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.802433 15443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:52.803223 15458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.803397 15443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:52.803481 15443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "a6ecf946296d46b59e9d5f7ac179334c"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "b5d5657321edb6fdf1c8800d798ec653"
server_key_iv: "3739dcf25aff7f3c78ff29f599e1a273"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.803545 15443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:52.816608 15443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:52.818737 15443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:52.818878 15443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:52.819095 15443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:52.819392 15443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:52.819425 15443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.819456 15443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:52.819479 15443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.825359 15443 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:39469
I20260925 22:00:52.825453 15571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:39469 every 8 connection(s)
I20260925 22:00:52.825744 15443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:52.828725 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15443
I20260925 22:00:52.828819 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:52.828994 9665 external_mini_cluster.cc:1428] Setting key 9fff4f590bc79cd7dbe2aa2753a4ec79
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:52.836213 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.826527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:58849 (local address 127.9.112.126:41165)
0925 22:00:52.826583 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:52.826585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:52.826795 (+ 210us) server_negotiation.cc:407] Connection header received
0925 22:00:52.827696 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:52.827698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:52.827709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:52.827764 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:52.828454 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.829008 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.830395 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.830592 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.830650 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.833048 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:52.833050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:52.833051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:52.833059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:52.834610 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.835106 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.835107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.835107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.835178 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.835453 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.835454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.835454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.835613 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:52.835696 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:52.836020 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 22:00:52.836110 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:00:52.836325 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.826467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:41165 (local address 127.9.112.66:58849)
0925 22:00:52.826653 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:52.826692 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:52.827587 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:52.827823 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:52.827827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:52.828095 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:52.828334 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:52.828363 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.829125 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.829126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:52.830299 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:52.830303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.830681 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.830682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:52.830830 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.831462 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:52.831470 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:52.832937 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:52.834724 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:52.834731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:52.834733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:52.835026 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:52.835264 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:52.835265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:52.835265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:52.835385 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:52.835741 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:52.835747 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:52.835881 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 22:00:52.835977 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 22:00:52.836111 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":65,"threads_started":1}
I20260925 22:00:52.836885 15572 heartbeater.cc:347] Connected to a master server at 127.9.112.126:41165
I20260925 22:00:52.836992 15572 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
I20260925 22:00:52.837188 15572 heartbeater.cc:511] Master 127.9.112.126:41165 requested a full tablet report, sending...
I20260925 22:00:52.837584 15247 ts_manager.cc:194] Registered new tserver with Master: a6ecf946296d46b59e9d5f7ac179334c (127.9.112.66:39469)
I20260925 22:00:52.837937 15247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:58849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:00:52.855151 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41165
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:52.939911 15577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:52.940142 15577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:52.940181 15577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:52.941490 15577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:52.941546 15577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:52.941591 15577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:52.943224 15577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:52.943504 15577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:52.943746 15577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:52.948760 15582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.948838 15585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:52.948957 15583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:52.949249 15577 server_base.cc:1034] running on GCE node
I20260925 22:00:52.949473 15577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:52.949733 15577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:52.950888 15577 hybrid_clock.cc:648] HybridClock initialized: now 1790373652950875 us; error 24 us; skew 500 ppm
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:52.953456 15577 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:00:52.953933 15577 webserver.cc:460] Webserver started at http://127.9.112.67:45475/ using document root <none> and password file <none>
I20260925 22:00:52.954149 15577 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:52.954196 15577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:52.954325 15577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:52.955487 15577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "e031a4fefa25458ea595b865cc002ea4"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "ccc0abe5dd2a7f10a168488d4b6eaa37"
server_key_iv: "556e42cbd35169166cca9d1a85f74bf6"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.955825 15577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "e031a4fefa25458ea595b865cc002ea4"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "ccc0abe5dd2a7f10a168488d4b6eaa37"
server_key_iv: "556e42cbd35169166cca9d1a85f74bf6"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.957317 15577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:00:52.958149 15592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.958376 15577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:52.958458 15577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "e031a4fefa25458ea595b865cc002ea4"
format_stamp: "Formatted at 2026-09-25 22:00:52 on dist-test-slave-794d"
server_key: "ccc0abe5dd2a7f10a168488d4b6eaa37"
server_key_iv: "556e42cbd35169166cca9d1a85f74bf6"
server_key_version: "encryptionkey@0"
I20260925 22:00:52.958518 15577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:52.970649 15577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:52.973224 15577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:52.973376 15577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:52.973579 15577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:52.973856 15577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:52.973887 15577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.973909 15577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:52.973920 15577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:52.979399 15577 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:39617
I20260925 22:00:52.979454 15705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:39617 every 8 connection(s)
I20260925 22:00:52.979751 15577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:52.980182 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15577
I20260925 22:00:52.980284 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:52.980494 9665 external_mini_cluster.cc:1428] Setting key e6ea81cff700553a8b4262a76144801d
Sep 25 22:00:52 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373652, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:00:52.989629 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.980576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:56339 (local address 127.9.112.126:41165)
0925 22:00:52.980634 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:52.980637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:52.980904 (+ 267us) server_negotiation.cc:407] Connection header received
0925 22:00:52.981923 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:52.981924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:52.981934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:52.981982 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:52.982619 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.983162 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.984150 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.984388 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.984458 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.986778 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:52.986779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:52.986780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:52.986793 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:52.988176 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.988620 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.988622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.988622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.988682 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:52.988898 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:52.988899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:52.988899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:52.989016 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:52.989064 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:52.989479 (+ 415us) server_negotiation.cc:299] Negotiation successful
0925 22:00:52.989558 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:00:52.989804 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:52.980576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:41165 (local address 127.9.112.67:56339)
0925 22:00:52.980790 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:52.980834 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:52.981835 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:52.982037 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:52.982040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:52.982319 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:52.982519 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:52.982522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.983247 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.983248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:52.984070 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:52.984072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:52.984490 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:52.984491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:52.984623 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:52.985196 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:52.985206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:52.986672 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:52.988266 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:52.988273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:52.988275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:52.988552 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:52.988737 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:52.988738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:52.988738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:52.988846 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:52.989101 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:52.989107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:52.989262 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 22:00:52.989453 (+ 191us) client_negotiation.cc:240] Negotiation successful
0925 22:00:52.989601 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":142,"threads_started":1}
I20260925 22:00:52.990350 15706 heartbeater.cc:347] Connected to a master server at 127.9.112.126:41165
I20260925 22:00:52.990451 15706 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:52.990617 15706 heartbeater.cc:511] Master 127.9.112.126:41165 requested a full tablet report, sending...
I20260925 22:00:52.991034 15247 ts_manager.cc:194] Registered new tserver with Master: e031a4fefa25458ea595b865cc002ea4 (127.9.112.67:39617)
I20260925 22:00:52.991405 15247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:56339
I20260925 22:00:52.992357 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/test-admin.keytab.
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.008863 9665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 22:00:53.010465 15713 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:53.009694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.9.112.65:36265)
0925 22:00:53.009917 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:53.009922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:53.009932 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:53.009956 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:53.009958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:53.009971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:53.010033 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:53.010381 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58428: BlockingRecv error: recv got EOF from 127.0.0.1:58428 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":147,"threads_started":1}
W20260925 22:00:53.010655 9665 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc' of type FILE
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.011571 9665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.012434 9665 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:53.020236 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:53.012956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.9.112.65:36265)
0925 22:00:53.013013 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:53.013015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:53.013024 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:53.013081 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:53.013081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:53.013098 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:53.013144 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:53.013508 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:53.014253 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:53.014960 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:53.015352 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:53.015412 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:53.017578 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:53.017587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:53.017588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:53.017596 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:53.019171 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:53.019506 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:53.019509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:53.019509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:53.019568 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:53.019703 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:53.019703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:53.019704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:53.019868 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:53.020013 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:53.020056 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 22:00:53.020128 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260925 22:00:53.021962 15713 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:53.021500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.9.112.65:36265)
0925 22:00:53.021536 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:53.021539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:53.021579 (+ 40us) server_negotiation.cc:407] Connection header received
0925 22:00:53.021631 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:53.021632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:53.021640 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:53.021680 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:53.021927 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58442: BlockingRecv error: recv got EOF from 127.0.0.1:58442 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 22:00:53.022040 9665 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc' of type FILE
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.022957 9665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.023746 9665 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:53.030128 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:53.024291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.9.112.65:36265)
0925 22:00:53.024390 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:53.024391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:53.024404 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:00:53.024460 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:53.024461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:53.024468 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:53.024515 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:53.024878 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:53.025381 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:53.026123 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:53.026318 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:53.026397 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:53.028179 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:53.028180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:53.028180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:53.028187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:53.029273 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:53.029534 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:53.029535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:53.029535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:53.029580 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:53.029716 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:53.029716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:53.029717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:53.029792 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:53.029858 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:53.030003 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 22:00:53.030062 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260925 22:00:53.031826 15713 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:53.031290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.9.112.65:36265)
0925 22:00:53.031363 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:53.031365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:53.031374 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:53.031390 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:53.031391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:53.031399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:53.031454 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:53.031785 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58456: BlockingRecv error: recv got EOF from 127.0.0.1:58456 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260925 22:00:53.031944 9665 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790373628768857-9665-0/krb5kdc/krb5cc' of type FILE
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.032878 9665 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:53.033634 9665 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:00:53.040553 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:53.034141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.9.112.65:36265)
0925 22:00:53.034183 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:53.034185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:53.034240 (+ 55us) server_negotiation.cc:407] Connection header received
0925 22:00:53.034283 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:53.034284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:53.034291 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:53.034340 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:53.034724 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:53.035192 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:53.035965 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:53.036155 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:53.036210 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:53.038398 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:53.038400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:53.038400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:53.038408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:53.039531 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:53.039858 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:53.039859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:53.039859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:53.039915 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:53.040070 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:53.040071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:53.040071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:53.040173 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:53.040256 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:53.040383 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 22:00:53.040462 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:00:53.041132 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15309
I20260925 22:00:53.045930 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15443
I20260925 22:00:53.050081 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15577
I20260925 22:00:53.054082 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15215
2026-09-25T22:00:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1823 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15726](info): set up 2 sockets
Sep 25 22:00:53 dist-test-slave-794d krb5kdc[15726](info): commencing operation
krb5kdc: starting...
W20260925 22:00:54.040378 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab.
Entry for principal oryx/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab.
I20260925 22:00:54.160588 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:43433
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab
--principal=oryx/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:43433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:54.242340 15739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:54.242560 15739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:54.242583 15739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:54.243731 15739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:54.243772 15739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:54.243784 15739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:54.243795 15739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:54.243810 15739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:54.245362 15739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:43433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab
--principal=oryx/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:54.245575 15739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:54.245756 15739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:54.250408 15747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.250416 15744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.250416 15745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.250500 15739 server_base.cc:1034] running on GCE node
I20260925 22:00:54.250924 15739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:54.251220 15739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:54.252385 15739 hybrid_clock.cc:648] HybridClock initialized: now 1790373654252367 us; error 37 us; skew 500 ppm
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:54.254911 15739 init.cc:377] Logged in from keytab as oryx/127.9.112.126@KRBTEST.COM (short username oryx)
I20260925 22:00:54.255375 15739 webserver.cc:460] Webserver started at http://127.9.112.126:39817/ using document root <none> and password file <none>
I20260925 22:00:54.255604 15739 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:54.255652 15739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:54.255780 15739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:54.256882 15739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "291a9916b523445ba24ef4216bb3cb60"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "fe0ca43912eb6bd070a2086c6d4b9a60"
server_key_iv: "5e4d9ae64c3b07523457e4cb86663307"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.257207 15739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "291a9916b523445ba24ef4216bb3cb60"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "fe0ca43912eb6bd070a2086c6d4b9a60"
server_key_iv: "5e4d9ae64c3b07523457e4cb86663307"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.258486 15739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:54.259335 15754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.259553 15739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:54.259644 15739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "291a9916b523445ba24ef4216bb3cb60"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "fe0ca43912eb6bd070a2086c6d4b9a60"
server_key_iv: "5e4d9ae64c3b07523457e4cb86663307"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.259707 15739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:54.275360 15739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:54.277496 15739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:54.277639 15739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:54.281512 15739 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:43433
I20260925 22:00:54.281598 15806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:43433 every 8 connection(s)
I20260925 22:00:54.281920 15739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:54.282882 15807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:54.285226 15807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Bootstrap starting.
I20260925 22:00:54.285506 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15739
I20260925 22:00:54.285597 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:54.285772 9665 external_mini_cluster.cc:1428] Setting key d4268e1338c141fa5a8822464761b04a
I20260925 22:00:54.285845 15807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:54.286129 15807 log.cc:824] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:54.286921 15807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: No bootstrap required, opened a new log
I20260925 22:00:54.289088 15807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } }
I20260925 22:00:54.289269 15807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:54.289304 15807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291a9916b523445ba24ef4216bb3cb60, State: Initialized, Role: FOLLOWER
I20260925 22:00:54.289429 15807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } }
I20260925 22:00:54.289513 15807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:54.289546 15807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:54.289587 15807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:54.290194 15807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } }
I20260925 22:00:54.290292 15807 leader_election.cc:302] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 291a9916b523445ba24ef4216bb3cb60; no voters:
I20260925 22:00:54.290452 15807 leader_election.cc:288] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:54.290475 15812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:54.290576 15812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 LEADER]: Becoming Leader. State: Replica: 291a9916b523445ba24ef4216bb3cb60, State: Running, Role: LEADER
I20260925 22:00:54.290656 15812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } }
I20260925 22:00:54.290720 15807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:54.290983 15812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } }
I20260925 22:00:54.291124 15812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:54.291313 15812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291a9916b523445ba24ef4216bb3cb60. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } }
I20260925 22:00:54.291425 15812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:54.291785 15818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:54.292289 15818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:54.294077 15818 catalog_manager.cc:1347] Generated new cluster ID: c845a698900342038899bdde37dfb7c6
I20260925 22:00:54.294176 15818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:54.296579 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.286267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.9.112.126:43433)
0925 22:00:54.286436 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.286445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.286478 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:00:54.286961 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.286969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.287024 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.287157 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.287931 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.288877 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.289852 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.290249 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.290358 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.293133 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.293138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.293138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.293151 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.294866 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.295265 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.295268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.295269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.295388 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.295594 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.295595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.295595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.295928 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.296099 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.296171 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.296316 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260925 22:00:54.301983 15818 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:54.302728 15818 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:54.309707 15818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Generated new TSK 0
I20260925 22:00:54.309930 15818 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab.
Entry for principal oryx/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab.
I20260925 22:00:54.344004 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43433
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:54.426673 15833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:54.426889 15833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:54.426911 15833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:54.428068 15833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:54.428109 15833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:54.428135 15833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:54.429620 15833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:54.429863 15833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:54.430090 15833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:54.435221 15841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.435191 15838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.435175 15839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.435341 15833 server_base.cc:1034] running on GCE node
I20260925 22:00:54.435631 15833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:54.435860 15833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:54.437033 15833 hybrid_clock.cc:648] HybridClock initialized: now 1790373654437010 us; error 30 us; skew 500 ppm
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:54.439500 15833 init.cc:377] Logged in from keytab as oryx/127.9.112.65@KRBTEST.COM (short username oryx)
I20260925 22:00:54.439955 15833 webserver.cc:460] Webserver started at http://127.9.112.65:34693/ using document root <none> and password file <none>
I20260925 22:00:54.440148 15833 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:54.440186 15833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:54.440275 15833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:54.441298 15833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "2254d97d517b49adb1d96b2817bd98bc"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "b6894e5a641ef07f53100cd7710f9cd7"
server_key_iv: "ca8ad453f78c78baf4274108e7f4d998"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.441598 15833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "2254d97d517b49adb1d96b2817bd98bc"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "b6894e5a641ef07f53100cd7710f9cd7"
server_key_iv: "ca8ad453f78c78baf4274108e7f4d998"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.442744 15833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:54.443454 15848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.443606 15833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:54.443686 15833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "2254d97d517b49adb1d96b2817bd98bc"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "b6894e5a641ef07f53100cd7710f9cd7"
server_key_iv: "ca8ad453f78c78baf4274108e7f4d998"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.443745 15833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:54.456161 15833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:54.458094 15833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:54.458205 15833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:54.458403 15833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:54.458650 15833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:54.458679 15833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.458699 15833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:54.458710 15833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.464102 15833 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:42809
I20260925 22:00:54.464180 15961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:42809 every 8 connection(s)
I20260925 22:00:54.464582 15833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:54.469152 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15833
I20260925 22:00:54.469285 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:54.469453 9665 external_mini_cluster.cc:1428] Setting key 9ca364704e34da55793a26fd5b25b6fd
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:54.474121 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.465240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:54307 (local address 127.9.112.126:43433)
0925 22:00:54.465281 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.465283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.465488 (+ 205us) server_negotiation.cc:407] Connection header received
0925 22:00:54.465999 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.466000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.466010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.466059 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.467016 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.467509 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.468565 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.468784 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.468847 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.471154 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.471156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.471156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.471163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.472636 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.473088 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.473091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.473091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.473149 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.473403 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.473404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.473404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.473521 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.473580 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.473934 (+ 354us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.474028 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:00:54.474303 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.465227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:54307)
0925 22:00:54.465369 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.465405 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.465914 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.466116 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.466120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.466718 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.466908 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.466913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.467605 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.467606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.468467 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.468470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.468860 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.468861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.468998 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.469538 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.469548 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.471038 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.472737 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.472744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.472747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.472992 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.473210 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.473211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.473211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.473340 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.473579 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.473585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.473786 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.473904 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.474071 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260925 22:00:54.474814 15962 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43433
I20260925 22:00:54.474905 15962 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:54.475080 15962 heartbeater.cc:511] Master 127.9.112.126:43433 requested a full tablet report, sending...
I20260925 22:00:54.475567 15771 ts_manager.cc:194] Registered new tserver with Master: 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809)
I20260925 22:00:54.476171 15771 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:54307
WARNING: no policy specified for oryx/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab.
Entry for principal oryx/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab.
I20260925 22:00:54.495949 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43433
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:54.580265 15967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:54.580529 15967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:54.580565 15967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:54.582015 15967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:54.582099 15967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:54.582145 15967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:54.584414 15967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:54.584921 15967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:54.585286 15967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:54.591147 15973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.591149 15972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.591336 15967 server_base.cc:1034] running on GCE node
W20260925 22:00:54.591148 15975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.591614 15967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:54.591900 15967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:54.593062 15967 hybrid_clock.cc:648] HybridClock initialized: now 1790373654593046 us; error 25 us; skew 500 ppm
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:54.596019 15967 init.cc:377] Logged in from keytab as oryx/127.9.112.66@KRBTEST.COM (short username oryx)
I20260925 22:00:54.596666 15967 webserver.cc:460] Webserver started at http://127.9.112.66:46739/ using document root <none> and password file <none>
I20260925 22:00:54.596910 15967 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:54.596962 15967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:54.597088 15967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:54.598129 15967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "a083e388c8dd42708b9da73f667173fa"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "897cf09a753b633a436a04529800f925"
server_key_iv: "f8267b545d0fe93cbc1abecb921da687"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.598456 15967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "a083e388c8dd42708b9da73f667173fa"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "897cf09a753b633a436a04529800f925"
server_key_iv: "f8267b545d0fe93cbc1abecb921da687"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.599792 15967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:54.600562 15982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.600719 15967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:54.600809 15967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "a083e388c8dd42708b9da73f667173fa"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "897cf09a753b633a436a04529800f925"
server_key_iv: "f8267b545d0fe93cbc1abecb921da687"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.600919 15967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:54.618953 15967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:54.620790 15967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:54.620905 15967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:54.621095 15967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:54.621417 15967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:54.621451 15967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.621474 15967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:54.621484 15967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.626855 15967 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:44873
I20260925 22:00:54.626899 16095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:44873 every 8 connection(s)
I20260925 22:00:54.627199 15967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:54.631528 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 15967
I20260925 22:00:54.631636 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:54.631825 9665 external_mini_cluster.cc:1428] Setting key a356dab05f11491069402e78b22ad30f
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:54.636950 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.627999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:55903 (local address 127.9.112.126:43433)
0925 22:00:54.628065 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.628068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.628187 (+ 119us) server_negotiation.cc:407] Connection header received
0925 22:00:54.628735 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.628736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.628749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.628803 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.629789 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.630264 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.631224 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.631431 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.631501 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.633943 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.633945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.633945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.633953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.635366 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.635907 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.635908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.635909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.635972 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.636230 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.636231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.636232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.636405 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.636474 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.636760 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.636849 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:00:54.637131 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.627874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:55903)
0925 22:00:54.628052 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.628103 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.628623 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.628882 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.628885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.629452 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.629644 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.629649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.630345 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.630346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.631139 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.631141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.631529 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.631530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.631671 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.632245 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.632253 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.633845 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.635482 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.635490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.635492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.635803 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.636036 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.636037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.636037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.636161 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.636466 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.636473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.636630 (+ 157us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.636730 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.636902 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260925 22:00:54.637764 16096 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43433
I20260925 22:00:54.637883 16096 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:54.638088 16096 heartbeater.cc:511] Master 127.9.112.126:43433 requested a full tablet report, sending...
I20260925 22:00:54.638502 15771 ts_manager.cc:194] Registered new tserver with Master: a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873)
I20260925 22:00:54.638892 15771 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:55903
WARNING: no policy specified for oryx/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab.
Entry for principal oryx/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab.
I20260925 22:00:54.659373 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43433
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:54.739209 16101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:54.739444 16101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:54.739467 16101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:54.740676 16101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:54.740720 16101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:54.740746 16101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:54.742252 16101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:54.742448 16101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:54.742636 16101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:54.747493 16109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.747493 16107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.747586 16101 server_base.cc:1034] running on GCE node
W20260925 22:00:54.747498 16106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.747895 16101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:54.748096 16101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:54.749238 16101 hybrid_clock.cc:648] HybridClock initialized: now 1790373654749222 us; error 28 us; skew 500 ppm
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:54.751533 16101 init.cc:377] Logged in from keytab as oryx/127.9.112.67@KRBTEST.COM (short username oryx)
I20260925 22:00:54.751953 16101 webserver.cc:460] Webserver started at http://127.9.112.67:45283/ using document root <none> and password file <none>
I20260925 22:00:54.752151 16101 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:54.752197 16101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:54.752331 16101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:54.753403 16101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "36cedb52a1e74631bffaff0958d4dfa3"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "4072bc4904ea6c827c973190451f4cb8"
server_key_iv: "1bc19e6e3ae9813d0fe80e0a45ccb602"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.753728 16101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "36cedb52a1e74631bffaff0958d4dfa3"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "4072bc4904ea6c827c973190451f4cb8"
server_key_iv: "1bc19e6e3ae9813d0fe80e0a45ccb602"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.755100 16101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:00:54.755949 16116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.756160 16101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:54.756253 16101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "36cedb52a1e74631bffaff0958d4dfa3"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "4072bc4904ea6c827c973190451f4cb8"
server_key_iv: "1bc19e6e3ae9813d0fe80e0a45ccb602"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.756326 16101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:54.771302 16101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:54.773784 16101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:54.773917 16101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:54.774155 16101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:54.774459 16101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:54.774492 16101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.774516 16101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:54.774529 16101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.780320 16101 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:43545
I20260925 22:00:54.780400 16229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:43545 every 8 connection(s)
I20260925 22:00:54.780745 16101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:54.784441 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 16101
I20260925 22:00:54.784533 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:54.784765 9665 external_mini_cluster.cc:1428] Setting key 6a5896632ec046a856bd1bba6f356692
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:54.790274 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.781436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:59819 (local address 127.9.112.126:43433)
0925 22:00:54.781510 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.781512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.781696 (+ 184us) server_negotiation.cc:407] Connection header received
0925 22:00:54.782165 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.782166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.782175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.782226 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.783182 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.783694 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.784865 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.785179 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.785256 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.787455 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.787457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.787457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.787464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.788846 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.789291 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.789292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.789292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.789341 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.789649 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.789651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.789651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.789823 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.789885 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.790134 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.790198 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:00:54.790539 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.781418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:59819)
0925 22:00:54.781578 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.781617 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.782085 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.782285 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.782288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.782872 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.783078 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.783083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.783850 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.783851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.784769 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.784772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.785270 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.785270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.785414 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.785953 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.785962 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.787337 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.788962 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.788969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.788971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.789227 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.789423 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.789424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.789425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.789558 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.789890 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.789895 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.790025 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.790117 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.790269 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260925 22:00:54.791105 16230 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43433
I20260925 22:00:54.791203 16230 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:54.791359 16230 heartbeater.cc:511] Master 127.9.112.126:43433 requested a full tablet report, sending...
I20260925 22:00:54.791684 15771 ts_manager.cc:194] Registered new tserver with Master: 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545)
I20260925 22:00:54.792110 15771 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:59819
I20260925 22:00:54.796725 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.112.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.125.keytab.
Entry for principal oryx/127.9.112.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.125.keytab.
I20260925 22:00:54.823732 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:42739
--webserver_interface=127.9.112.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.125.keytab
--principal=oryx/127.9.112.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:43433,127.9.112.125:42739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:54.905524 16235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:54.905879 16235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:54.905947 16235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:54.907256 16235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:54.907299 16235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:54.907315 16235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:54.907328 16235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:54.907338 16235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:54.909188 16235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:43433,127.9.112.125:42739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.125.keytab
--principal=oryx/127.9.112.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.112.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
W20260925 22:00:54.909343 16235 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.112.126:43433,127.9.112.125:42739'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 22:00:54.909507 16235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:54.909772 16235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:54.915124 16243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.915205 16235 server_base.cc:1034] running on GCE node
W20260925 22:00:54.915148 16241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:54.915151 16240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:54.915555 16235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:54.915823 16235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:54.916980 16235 hybrid_clock.cc:648] HybridClock initialized: now 1790373654916962 us; error 28 us; skew 500 ppm
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:54.919688 16235 init.cc:377] Logged in from keytab as oryx/127.9.112.125@KRBTEST.COM (short username oryx)
I20260925 22:00:54.920158 16235 webserver.cc:460] Webserver started at http://127.9.112.125:44955/ using document root <none> and password file <none>
I20260925 22:00:54.920416 16235 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:54.920470 16235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:54.920596 16235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:54.921689 16235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/instance:
uuid: "d06122da75b64a4da086ec98076cafb9"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "e32a1b6b6d1c96283489206051e777ac"
server_key_iv: "d651f8cd8989cea1c32cd42e5de868b1"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.922031 16235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal/instance:
uuid: "d06122da75b64a4da086ec98076cafb9"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "e32a1b6b6d1c96283489206051e777ac"
server_key_iv: "d651f8cd8989cea1c32cd42e5de868b1"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.923426 16235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:54.924153 16250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:54.924317 16235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:54.924427 16235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
uuid: "d06122da75b64a4da086ec98076cafb9"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "e32a1b6b6d1c96283489206051e777ac"
server_key_iv: "d651f8cd8989cea1c32cd42e5de868b1"
server_key_version: "encryptionkey@0"
I20260925 22:00:54.924495 16235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:54.934006 16235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:54.936895 16235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:54.937052 16235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:54.941071 16235 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.125:42739
I20260925 22:00:54.941110 16302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.125:42739 every 8 connection(s)
I20260925 22:00:54.941466 16235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
I20260925 22:00:54.942453 16303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:54.944067 16303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:00:54.948730 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 16235
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.125@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:54.954492 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.944959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.9.112.126:43433)
0925 22:00:54.945038 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.945040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.945174 (+ 134us) server_negotiation.cc:407] Connection header received
0925 22:00:54.945685 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.945686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.945697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.945764 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.946479 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.947016 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.948072 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.948281 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.948385 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.951138 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.951140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.951140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.951150 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.952805 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.953310 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.953311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.953311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.953371 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.953659 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.953660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.953660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.953868 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.953941 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.954304 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.954397 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
I20260925 22:00:54.954651 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.944846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:55228)
0925 22:00:54.945027 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.945072 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.945538 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.945831 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.945835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.946143 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.946374 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.946379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.947105 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.947105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.947985 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.947987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.948425 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.948426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.948563 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.949395 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.949407 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.951030 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.952917 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.952925 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.952927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.953220 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.953456 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.953457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.953457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.953587 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.953960 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.953966 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.954107 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.954251 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.954421 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260925 22:00:54.955359 16303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:00:54.957854 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.949260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.9.112.125:42739)
0925 22:00:54.949440 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.949445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.949465 (+ 20us) server_negotiation.cc:407] Connection header received
0925 22:00:54.949489 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.949492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.949513 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.949601 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.950179 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.951030 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.951998 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.952511 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.952581 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.955090 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.955097 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.955097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.955104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.956766 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.957108 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.957111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.957111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.957174 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.957359 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.957360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.957360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.957519 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.957679 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.957717 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.957779 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.125@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
I20260925 22:00:54.962551 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.955616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:48192)
0925 22:00:54.955672 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.955752 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.955792 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.955996 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.955996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.956158 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.956201 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.956203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.956905 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.956906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.957553 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.957554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.958026 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.958026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.958071 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.958424 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.958429 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.959553 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.961549 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.961550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.961550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.961772 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.962003 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.962004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.962005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.962065 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.962306 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.962306 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.962343 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.962395 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.962477 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:00:54.962687 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.955678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.9.112.125:42739)
0925 22:00:54.955876 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.955878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.955886 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:54.955904 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.955904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.955916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.955985 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.956280 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.956835 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.957616 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.957919 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.957985 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.959643 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.959644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.959645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.959651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.961460 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.961853 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.961854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.961854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.961913 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.962127 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.962127 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.962127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.962259 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.962311 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.962417 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.962555 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260925 22:00:54.964041 16303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Bootstrap starting.
I20260925 22:00:54.964687 16303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:54.964913 16303 log.cc:824] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:54.965739 16303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: No bootstrap required, opened a new log
I20260925 22:00:54.967149 16303 raft_consensus.cc:348] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } }
I20260925 22:00:54.967293 16303 raft_consensus.cc:374] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:54.967316 16303 raft_consensus.cc:729] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06122da75b64a4da086ec98076cafb9, State: Initialized, Role: FOLLOWER
I20260925 22:00:54.967420 16303 consensus_queue.cc:260] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } }
I20260925 22:00:54.967712 16303 sys_catalog.cc:564] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:54.967694 16310 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } } }
I20260925 22:00:54.967792 16310 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:00:54.969270 16321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:00:54.969332 16321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:00:54.970923 16235 master_runner.cc:428] Detected that this master d06122da75b64a4da086ec98076cafb9 is joining an existing cluster
I20260925 22:00:54.970971 16235 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.112.125:42739
I20260925 22:00:54.978493 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.972322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.9.112.125:42739)
0925 22:00:54.972386 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.972388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.972749 (+ 361us) server_negotiation.cc:407] Connection header received
0925 22:00:54.972808 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.972809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.972818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.972861 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.973125 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.973638 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.974354 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.974568 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.974623 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.975650 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.975651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.975651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.975658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.977443 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.977749 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.977750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.977750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.977802 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.977992 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.977993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.977993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.978129 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.978166 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.978347 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.978425 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:00:54.978788 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.972128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:48206)
0925 22:00:54.972608 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.972618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.972745 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.972891 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.972891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.973026 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.973064 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.973065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.973709 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.973710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.974287 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.974289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.974681 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.974682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.974734 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.975186 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.975191 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.975576 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.977520 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.977521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.977521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.977691 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.977863 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.977864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.977864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.977931 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.978195 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.978196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.978226 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.978610 (+ 384us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.978703 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"spinlock_wait_cycles":8576,"thread_start_us":102,"threads_started":1}
I20260925 22:00:54.978942 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.972325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.9.112.126:43433)
0925 22:00:54.972411 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.972413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.972424 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:00:54.972450 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.972450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.972461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.972531 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.972860 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.973634 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.974751 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.975040 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.975080 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.976070 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.976071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.976072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.976079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.977758 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.978107 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.978109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.978109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.978182 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.978363 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.978364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.978364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.978504 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.978566 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.978768 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.978847 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 22:00:54.979272 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.972125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:55232)
0925 22:00:54.972289 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.972301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.972386 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.972534 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.972535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.972701 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.972796 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.972799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.973751 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.973752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.974649 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.974652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.975154 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.975155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.975221 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.975590 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.975595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.975983 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.977846 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.977846 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.977847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.978035 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.978250 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.978251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.978251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.978308 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.978594 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.978595 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.978650 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.979064 (+ 414us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.979166 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
Sep 25 22:00:54 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.126@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
I20260925 22:00:54.989791 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.981831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:48210)
0925 22:00:54.981957 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:54.982003 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:54.982075 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:54.982231 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:54.982234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:54.982527 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:54.982588 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.982590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.983597 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.983599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.984271 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:54.984273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.984815 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.984816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:54.984891 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.985412 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:54.985417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:54.987004 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:54.988801 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.988809 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.988811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.989026 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.989208 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:54.989209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:54.989209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:54.989264 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:54.989497 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:54.989503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:54.989547 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 22:00:54.989607 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 22:00:54.989703 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260925 22:00:54.989823 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:54.981910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.9.112.125:42739)
0925 22:00:54.981955 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:54.981957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:54.982065 (+ 108us) server_negotiation.cc:407] Connection header received
0925 22:00:54.982156 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:54.982157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:54.982167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:54.982221 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:54.982711 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.983500 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.984382 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:54.984687 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:54.984756 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:54.987090 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:54.987092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:54.987093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:54.987100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:54.988706 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.989090 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.989091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.989091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.989151 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:54.989312 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:54.989312 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:54.989312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:54.989443 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:54.989509 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:54.989643 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 22:00:54.989729 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:00:54.990437 15771 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.112.125:42739: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d06122da75b64a4da086ec98076cafb9"
member_type: NON_VOTER
last_known_addr {
host: "127.9.112.125"
port: 42739
}
attrs {
promote: true
}
}
dest_uuid: "291a9916b523445ba24ef4216bb3cb60"
cas_config_opid_index: -1
I20260925 22:00:54.990746 15771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } }
I20260925 22:00:54.991206 16331 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d06122da75b64a4da086ec98076cafb9 (127.9.112.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } }
I20260925 22:00:54.991576 16277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:54.992295 16331 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.112.125:42739
I20260925 22:00:54.992508 16277 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Refusing update from remote peer 291a9916b523445ba24ef4216bb3cb60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 22:00:54.992591 16330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } }
I20260925 22:00:54.992659 16330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:54.992671 16235 master.cc:537] Master@127.9.112.125:42739 shutting down...
I20260925 22:00:54.992782 16331 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 22:00:54.993561 15757 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 22:00:54.993966 15757 consensus_peers.cc:487] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 -> Peer d06122da75b64a4da086ec98076cafb9 (127.9.112.125:42739): Couldn't send request to peer d06122da75b64a4da086ec98076cafb9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 22:00:54.995520 16235 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:54.995612 16235 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:54.995659 16235 tablet_replica.cc:330] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: stopping tablet replica
I20260925 22:00:55.008905 16235 master.cc:552] Master@127.9.112.125:42739 shutdown complete.
I20260925 22:00:55.008967 16235 master_runner.cc:305] Clearing existing system tablet
I20260925 22:00:55.009094 16235 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.009141 16235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.009886 16235 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:55.010360 16335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.010483 16235 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.010583 16235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.010622 16235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
uuid: "d06122da75b64a4da086ec98076cafb9"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "e32a1b6b6d1c96283489206051e777ac"
server_key_iv: "d651f8cd8989cea1c32cd42e5de868b1"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.011246 16235 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:55.013494 16235 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 22:00:55.013566 16235 log.cc:1196] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 22:00:55.013799 16235 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Deleting consensus metadata
I20260925 22:00:55.013903 16235 master_runner.cc:315] Copying system tablet from 127.9.112.126:43433
I20260925 22:00:55.014510 16235 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.112.126:43433
I20260925 22:00:55.021414 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.014871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:55234)
0925 22:00:55.014996 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.015009 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.015075 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.015207 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.015208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.015362 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.015429 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.015431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.016484 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.016486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.017089 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.017092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.017579 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.017580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.017643 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.018156 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.018161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.018474 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.020294 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.020295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.020295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.020528 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.020766 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.020768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.020768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.020869 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.021130 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.021131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.021174 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.021219 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.021315 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260925 22:00:55.021448 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.014970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.9.112.126:43433)
0925 22:00:55.015046 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.015048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.015065 (+ 17us) server_negotiation.cc:407] Connection header received
0925 22:00:55.015125 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.015126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.015136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.015201 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.015492 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.016272 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.017177 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.017486 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.017560 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.018557 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.018558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.018558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.018565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.020203 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.020606 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.020612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.020613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.020678 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.020931 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.020932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.020932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.021072 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.021133 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.021256 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.021338 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 22:00:55.021848 15791 tablet_copy_service.cc:140] P 291a9916b523445ba24ef4216bb3cb60: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d06122da75b64a4da086ec98076cafb9 ({username='oryx', principal='oryx/127.9.112.125@KRBTEST.COM'} at 127.0.0.1:55234)
I20260925 22:00:55.021965 15791 tablet_copy_service.cc:161] P 291a9916b523445ba24ef4216bb3cb60: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d06122da75b64a4da086ec98076cafb9 at {username='oryx', principal='oryx/127.9.112.125@KRBTEST.COM'} at 127.0.0.1:55234: session id = d06122da75b64a4da086ec98076cafb9-00000000000000000000000000000000
I20260925 22:00:55.022707 15791 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 22:00:55.023342 16235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 22:00:55.024278 16235 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: tablet copy: Starting download of 0 data blocks...
I20260925 22:00:55.024478 16235 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: tablet copy: Starting download of 1 WAL segments...
I20260925 22:00:55.025252 16235 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 22:00:55.026046 15791 tablet_copy_service.cc:342] P 291a9916b523445ba24ef4216bb3cb60: Request end of tablet copy session d06122da75b64a4da086ec98076cafb9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.112.125@KRBTEST.COM'} at 127.0.0.1:55234
I20260925 22:00:55.026094 15791 tablet_copy_service.cc:433] P 291a9916b523445ba24ef4216bb3cb60: ending tablet copy session d06122da75b64a4da086ec98076cafb9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d06122da75b64a4da086ec98076cafb9
I20260925 22:00:55.028409 16235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:55.029639 16345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.029639 16342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.029855 16343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.029891 16235 server_base.cc:1034] running on GCE node
I20260925 22:00:55.030000 16235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:55.030139 16235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:55.031327 16235 hybrid_clock.cc:648] HybridClock initialized: now 1790373655031317 us; error 24 us; skew 500 ppm
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:55.032994 16235 init.cc:377] Logged in from keytab as oryx/127.9.112.125@KRBTEST.COM (short username oryx)
I20260925 22:00:55.033313 16235 webserver.cc:460] Webserver started at http://127.9.112.125:39221/ using document root <none> and password file <none>
I20260925 22:00:55.033425 16235 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.033453 16235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.034107 16235 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:00:55.034598 16352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.034739 16235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.034804 16235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
uuid: "d06122da75b64a4da086ec98076cafb9"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "e32a1b6b6d1c96283489206051e777ac"
server_key_iv: "d651f8cd8989cea1c32cd42e5de868b1"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.034860 16235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.056882 16235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:55.058772 16235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:55.062371 16235 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.125:42739
I20260925 22:00:55.062671 16404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.125:42739 every 8 connection(s)
I20260925 22:00:55.062886 16235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
I20260925 22:00:55.063686 16405 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:00:55.064079 16405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Bootstrap starting.
I20260925 22:00:55.066576 16405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:00:55.066809 16405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: Bootstrap complete.
I20260925 22:00:55.066992 16405 raft_consensus.cc:348] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } }
I20260925 22:00:55.067060 16405 raft_consensus.cc:729] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d06122da75b64a4da086ec98076cafb9, State: Initialized, Role: LEARNER
I20260925 22:00:55.067094 16405 consensus_queue.cc:260] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } }
I20260925 22:00:55.067260 16405 sys_catalog.cc:564] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:55.067263 16408 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } }
I20260925 22:00:55.067359 16408 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: LEARNER
I20260925 22:00:55.068638 16419 catalog_manager.cc:1259] Loaded cluster ID: c845a698900342038899bdde37dfb7c6
I20260925 22:00:55.068686 16419 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: loading cluster ID for follower catalog manager: success
I20260925 22:00:55.069607 16419 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: acquiring CA information for follower catalog manager: success
I20260925 22:00:55.069873 16419 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:00:55.115782 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.109894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.9.112.125:42739)
0925 22:00:55.110018 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.110019 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.110026 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:00:55.110047 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.110048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.110056 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.110114 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.110652 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.111369 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.112088 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.112280 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.112369 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.113464 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.113466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.113466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.113473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.114745 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.115079 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.115080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.115081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.115133 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.115296 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.115297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.115297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.115413 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.115503 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.115596 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.115684 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.9.112.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.124.keytab.
Entry for principal oryx/127.9.112.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.124.keytab.
I20260925 22:00:55.145066 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:34087
--webserver_interface=127.9.112.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.124.keytab
--principal=oryx/127.9.112.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:55.237972 16423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:55.238207 16423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:55.238230 16423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:55.239478 16423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:55.239527 16423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:55.239540 16423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:55.239552 16423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:55.239562 16423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:55.241259 16423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.124.keytab
--principal=oryx/127.9.112.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.112.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:55.241636 16423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:55.241982 16423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:55.247265 16429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.247265 16431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.247268 16428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.247589 16423 server_base.cc:1034] running on GCE node
I20260925 22:00:55.247908 16423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:55.248286 16423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:55.249454 16423 hybrid_clock.cc:648] HybridClock initialized: now 1790373655249433 us; error 37 us; skew 500 ppm
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:55.252539 16423 init.cc:377] Logged in from keytab as oryx/127.9.112.124@KRBTEST.COM (short username oryx)
I20260925 22:00:55.253111 16423 webserver.cc:460] Webserver started at http://127.9.112.124:42695/ using document root <none> and password file <none>
I20260925 22:00:55.253322 16423 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.253372 16423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.253468 16423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:55.254563 16423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/instance:
uuid: "a8eb490f50c1409db6576b612a354b5c"
format_stamp: "Formatted at 2026-09-25 22:00:55 on dist-test-slave-794d"
server_key: "5a9b93e821f1b8f22ae92dac8447474c"
server_key_iv: "0c35a620a29a43b97e8f9eccb08c9fb0"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.254936 16423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal/instance:
uuid: "a8eb490f50c1409db6576b612a354b5c"
format_stamp: "Formatted at 2026-09-25 22:00:55 on dist-test-slave-794d"
server_key: "5a9b93e821f1b8f22ae92dac8447474c"
server_key_iv: "0c35a620a29a43b97e8f9eccb08c9fb0"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.256454 16423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:55.257572 16438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.257954 16423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:55.258049 16423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
uuid: "a8eb490f50c1409db6576b612a354b5c"
format_stamp: "Formatted at 2026-09-25 22:00:55 on dist-test-slave-794d"
server_key: "5a9b93e821f1b8f22ae92dac8447474c"
server_key_iv: "0c35a620a29a43b97e8f9eccb08c9fb0"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.258136 16423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.266873 16423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:55.269372 16423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:55.269491 16423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:55.273351 16423 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.124:34087
I20260925 22:00:55.273439 16490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.124:34087 every 8 connection(s)
I20260925 22:00:55.273749 16423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
I20260925 22:00:55.274794 16491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:55.276310 16491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:00:55.280941 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 16423
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.124@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.112.124@KRBTEST.COM
I20260925 22:00:55.288937 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.277287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.9.112.126:43433)
0925 22:00:55.277343 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.277345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.277380 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:00:55.277905 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.277907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.277917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.277978 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.278677 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.279189 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.280277 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.280533 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.280593 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.285448 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.285449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.285450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.285457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.286938 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.287504 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.287507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.287508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.287658 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.288063 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.288064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.288065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.288259 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.288380 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.288730 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.288837 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:00:55.289004 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.277026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:55244)
0925 22:00:55.277232 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.277280 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.277780 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.278081 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.278086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.278362 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.278563 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.278568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.279286 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.279286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.280176 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.280179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.280645 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.280646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.280898 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.283662 (+ 2764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.283672 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.285333 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.287026 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.287036 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.287038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.287352 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.287825 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.287826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.287826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.287959 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.288370 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.288377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.288531 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.288632 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.288768 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260925 22:00:55.289546 16491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:00:55.291047 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.281752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.9.112.124:34087)
0925 22:00:55.281930 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.281935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.281950 (+ 15us) server_negotiation.cc:407] Connection header received
0925 22:00:55.281969 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.281970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.281990 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.282057 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.282635 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.283334 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.284026 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.284425 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.284492 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.287116 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.287123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.287123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.287129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.289278 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.289664 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.289668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.289668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.289745 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.290233 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.290234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.290234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.290475 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.290619 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.290812 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.290945 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.124@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
I20260925 22:00:55.297614 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.289816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:48250)
0925 22:00:55.289850 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.289861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.289890 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.290228 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.290229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.290375 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.290424 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.290426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.291539 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.291540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.292170 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.292172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.292574 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.292575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.292635 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.293196 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.293200 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.294488 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.296639 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.296641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.296641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.296828 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.297033 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.297034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.297034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.297080 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.297332 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.297333 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.297382 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.297451 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.297539 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 22:00:55.297667 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.289955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.9.112.125:42739)
0925 22:00:55.290065 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.290067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.290081 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:00:55.290106 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.290107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.290122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.290196 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.290540 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.291457 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.292257 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.292511 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.292573 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.294603 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.294605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.294605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.294613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.296512 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.296915 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.296916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.296917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.296977 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.297122 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.297122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.297122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.297250 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.297301 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.297509 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.297594 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 22:00:55.298125 16491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.124@KRBTEST.COM for oryx/127.9.112.124@KRBTEST.COM
I20260925 22:00:55.305039 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.298463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.9.112.124:34087)
0925 22:00:55.298504 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.298505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.298513 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:55.298547 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.298547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.298558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.298599 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.298872 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.299387 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.299992 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.300202 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.300255 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.301984 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.301985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.301986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.301993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.303993 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.304319 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.304320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.304321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.304400 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.304548 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.304549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.304549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.304667 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.304719 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.304838 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.304923 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":26}
I20260925 22:00:55.305050 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.298412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.0.0.1:55170)
0925 22:00:55.298456 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.298466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.298503 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.298600 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.298601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.298723 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.298757 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.298759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.299457 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.299457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.299925 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.299927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.300257 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.300258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.300300 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.300770 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.300775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.301899 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.304093 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.304094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.304094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.304259 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.304451 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.304451 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.304451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.304500 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.304717 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.304717 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.304755 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.304809 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.304969 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":68}
I20260925 22:00:55.306694 16491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Bootstrap starting.
I20260925 22:00:55.307510 16491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:55.307768 16491 log.cc:824] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:55.308640 16491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: No bootstrap required, opened a new log
I20260925 22:00:55.310101 16491 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } }
I20260925 22:00:55.310242 16491 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:55.310274 16491 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8eb490f50c1409db6576b612a354b5c, State: Initialized, Role: FOLLOWER
I20260925 22:00:55.310412 16491 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } }
I20260925 22:00:55.310721 16491 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:55.310695 16496 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } } }
I20260925 22:00:55.310788 16496 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:00:55.312307 16507 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:00:55.312423 16507 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:00:55.314822 16423 master_runner.cc:428] Detected that this master a8eb490f50c1409db6576b612a354b5c is joining an existing cluster
I20260925 22:00:55.314893 16423 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.112.124:34087
I20260925 22:00:55.341862 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.316129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.9.112.124:34087)
0925 22:00:55.316172 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.316173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.317231 (+ 1058us) server_negotiation.cc:407] Connection header received
0925 22:00:55.317374 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.317375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.317391 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.317467 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.318900 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.319645 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.320521 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.320824 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.320888 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.321893 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.321894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.321894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.321899 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.323304 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.341048 (+ 17744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.341049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.341050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.341120 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.341296 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.341297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.341297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.341443 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.341497 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.341662 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.341755 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":20}
I20260925 22:00:55.342221 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.316528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.9.112.125:42739)
0925 22:00:55.316578 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.316580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.317233 (+ 653us) server_negotiation.cc:407] Connection header received
0925 22:00:55.317526 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.317527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.317538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.317590 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.318014 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.318796 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.319592 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.319937 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.320006 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.321120 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.321121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.321122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.321130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.322439 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.341049 (+ 18610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.341052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.341052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.341161 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.341418 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.341420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.341420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.341555 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.341614 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.341820 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.341922 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:00:55.342263 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.316224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.0.0.1:55184)
0925 22:00:55.317012 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.317020 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.317257 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.317489 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.317490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.317814 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.317931 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.317934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.319731 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.319731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.320438 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.320441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.320935 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.320935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.320996 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.321392 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.321397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.321799 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.323386 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.323387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.323388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.340929 (+ 17541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.341186 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.341187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.341187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.341240 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.341522 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.341523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.341559 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.341992 (+ 433us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.342088 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":6,"thread_start_us":994,"threads_started":1}
I20260925 22:00:55.342590 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.316121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:48256)
0925 22:00:55.316701 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.316711 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.317258 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.317616 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.317617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.317812 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.317934 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.317936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.318885 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.318886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.319502 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.319504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.320023 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.320025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.320093 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.320670 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.320675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.320994 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.322510 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.322512 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.322512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.340870 (+ 18358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.341246 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.341247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.341247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.341321 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.341652 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.341653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.341710 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.342386 (+ 676us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.342495 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":15,"thread_start_us":203,"threads_started":1}
I20260925 22:00:55.342994 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15833
I20260925 22:00:55.346354 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.316161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.9.112.126:43433)
0925 22:00:55.316207 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.316209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.328586 (+ 12377us) server_negotiation.cc:407] Connection header received
0925 22:00:55.328636 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.328637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.328651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.328721 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.329339 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.329872 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.331000 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.331193 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.331264 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.341351 (+ 10087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.341353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.341353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.341380 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.345121 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.345485 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.345486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.345487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.345540 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.345808 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.345810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.345810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.345956 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.346024 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.346163 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.346249 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:00:55.346381 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.316900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:55246)
0925 22:00:55.328456 (+ 11556us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.328473 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.328565 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.328779 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.328780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.329045 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.329122 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.329125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.329961 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.329962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.330885 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.330888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.331272 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.331273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.331340 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.331849 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.331854 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.341239 (+ 9385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.345214 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.345215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.345216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.345405 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.345603 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.345603 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.345604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.345666 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.346026 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.346027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.346073 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.346126 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.346256 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11519,"thread_start_us":59,"threads_started":1}
I20260925 22:00:55.349504 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:42809
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:43433
--webserver_port=34693
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.126@KRBTEST.COM for oryx/127.9.112.124@KRBTEST.COM
I20260925 22:00:55.355983 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.348563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.0.0.1:55200)
0925 22:00:55.348602 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.348612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.348649 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.349018 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.349021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.349294 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.349531 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.349536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.350676 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.350678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.351505 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.351511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.351862 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.351863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.351934 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.352376 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.352382 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.353604 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.355061 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.355062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.355063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.355229 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.355395 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.355395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.355396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.355452 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.355666 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.355666 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.355715 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.355781 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.355888 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 22:00:55.355981 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.348654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.9.112.124:34087)
0925 22:00:55.348704 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.348706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.348781 (+ 75us) server_negotiation.cc:407] Connection header received
0925 22:00:55.348819 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.348820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.348832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.348898 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.349777 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.350535 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.351602 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.351795 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.351853 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.353707 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.353708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.353709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.353716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.354981 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.355291 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.355292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.355292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.355345 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.355498 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.355498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.355499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.355613 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.355665 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.355804 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.355879 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:00:55.356861 15771 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.112.124:34087: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a8eb490f50c1409db6576b612a354b5c"
member_type: NON_VOTER
last_known_addr {
host: "127.9.112.124"
port: 34087
}
attrs {
promote: true
}
}
dest_uuid: "291a9916b523445ba24ef4216bb3cb60"
cas_config_opid_index: 5
I20260925 22:00:55.357111 15771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } }
I20260925 22:00:55.357443 16330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a8eb490f50c1409db6576b612a354b5c (127.9.112.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.357915 16465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:55.358574 16330 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.112.124:34087
I20260925 22:00:55.358734 16331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.358824 16331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:55.358846 16465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Refusing update from remote peer 291a9916b523445ba24ef4216bb3cb60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 22:00:55.359102 16331 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:55.359371 16423 master.cc:537] Master@127.9.112.124:34087 shutting down...
W20260925 22:00:55.359907 15758 consensus_peers.cc:487] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 -> Peer a8eb490f50c1409db6576b612a354b5c (127.9.112.124:34087): Couldn't send request to peer a8eb490f50c1409db6576b612a354b5c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 22:00:55.364298 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.357587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.9.112.125:42739)
0925 22:00:55.357775 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.357776 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.357892 (+ 116us) server_negotiation.cc:407] Connection header received
0925 22:00:55.357916 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.357917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.357938 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.357995 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:55.358174 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.360623 (+ 2449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.362657 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.363289 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.363347 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.363416 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.364182 (+ 766us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.364229 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260925 22:00:55.364562 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.357533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:48272)
0925 22:00:55.357803 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.357814 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.357851 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.358055 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.358056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.358058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:55.358104 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.358105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.360727 (+ 2622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.360728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.362538 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.362543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.363452 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.363453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.363520 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.363578 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.363624 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.363668 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260925 22:00:55.364825 16379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 LEARNER]: Refusing update from remote peer 291a9916b523445ba24ef4216bb3cb60: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 22:00:55.365029 16331 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 22:00:55.365092 16423 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:55.365173 16423 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:55.365216 16423 tablet_replica.cc:330] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: stopping tablet replica
I20260925 22:00:55.365612 16379 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a8eb490f50c1409db6576b612a354b5c (127.9.112.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.366593 16408 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291a9916b523445ba24ef4216bb3cb60. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.366662 16408 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: LEARNER
I20260925 22:00:55.366693 16408 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: NON_VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: true } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.366731 16408 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: LEARNER
I20260925 22:00:55.367314 16331 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: attempting to promote NON_VOTER d06122da75b64a4da086ec98076cafb9 to VOTER
I20260925 22:00:55.367549 16331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } }
I20260925 22:00:55.368031 16379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 LEARNER]: Refusing update from remote peer 291a9916b523445ba24ef4216bb3cb60: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 22:00:55.368275 16330 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260925 22:00:55.368444 15758 connection.cc:535] client connection to 127.9.112.124:34087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:00:55.368814 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.368706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.0.0.1:55206)
0925 22:00:55.368746 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.368786 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 22:00:55.368928 16330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d06122da75b64a4da086ec98076cafb9 (127.9.112.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.369059 16379 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d06122da75b64a4da086ec98076cafb9 (127.9.112.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.369774 16330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.369843 16330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:55.369887 16330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.369935 16330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:55.369972 16518 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.370059 16518 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:00:55.370069 15758 consensus_peers.cc:487] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 -> Peer a8eb490f50c1409db6576b612a354b5c (127.9.112.124:34087): Couldn't send request to peer a8eb490f50c1409db6576b612a354b5c. Status: Network error: Client connection negotiation failed: client connection to 127.9.112.124:34087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 22:00:55.371279 16423 master.cc:552] Master@127.9.112.124:34087 shutdown complete.
I20260925 22:00:55.371318 16423 master_runner.cc:305] Clearing existing system tablet
I20260925 22:00:55.371446 16423 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.371488 16423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.372054 16423 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:00:55.372618 16521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.372761 16423 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.372809 16423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.372831 16423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
uuid: "a8eb490f50c1409db6576b612a354b5c"
format_stamp: "Formatted at 2026-09-25 22:00:55 on dist-test-slave-794d"
server_key: "5a9b93e821f1b8f22ae92dac8447474c"
server_key_iv: "0c35a620a29a43b97e8f9eccb08c9fb0"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.373545 16423 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:55.375123 16423 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 22:00:55.375191 16423 log.cc:1196] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 22:00:55.375409 16423 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Deleting consensus metadata
I20260925 22:00:55.375516 16423 master_runner.cc:315] Copying system tablet from 127.9.112.126:43433
I20260925 22:00:55.376075 16423 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: tablet copy: Beginning tablet copy session from remote peer at address 127.9.112.126:43433
I20260925 22:00:55.383205 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.376443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.9.112.126:43433)
0925 22:00:55.376489 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.376491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.376564 (+ 73us) server_negotiation.cc:407] Connection header received
0925 22:00:55.376635 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.376636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.376646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.376693 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.376980 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.377474 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.378214 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.378410 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.378464 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.379470 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.379471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.379472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.379479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.381924 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.382315 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.382317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.382317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.382393 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.382600 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.382601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.382601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.382756 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.382817 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.383011 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.383101 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:00:55.383786 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.376395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:55252)
0925 22:00:55.376511 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.376521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.376578 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.376696 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.376697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.376862 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.376913 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.376915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.377547 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.377548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.378141 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.378143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.378471 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.378472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.378532 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.378981 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.378986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.379372 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.382020 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.382022 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.382022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.382260 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.382461 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.382462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.382463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.382553 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.382844 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.382845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.382896 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.383355 (+ 459us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.383485 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260925 22:00:55.384020 15791 tablet_copy_service.cc:140] P 291a9916b523445ba24ef4216bb3cb60: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a8eb490f50c1409db6576b612a354b5c ({username='oryx', principal='oryx/127.9.112.124@KRBTEST.COM'} at 127.0.0.1:55252)
I20260925 22:00:55.384107 15791 tablet_copy_service.cc:161] P 291a9916b523445ba24ef4216bb3cb60: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a8eb490f50c1409db6576b612a354b5c at {username='oryx', principal='oryx/127.9.112.124@KRBTEST.COM'} at 127.0.0.1:55252: session id = a8eb490f50c1409db6576b612a354b5c-00000000000000000000000000000000
I20260925 22:00:55.384812 15791 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 22:00:55.385324 16423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 22:00:55.386747 16423 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: tablet copy: Starting download of 0 data blocks...
I20260925 22:00:55.386889 16423 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: tablet copy: Starting download of 1 WAL segments...
I20260925 22:00:55.387522 16423 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 22:00:55.388406 15791 tablet_copy_service.cc:342] P 291a9916b523445ba24ef4216bb3cb60: Request end of tablet copy session a8eb490f50c1409db6576b612a354b5c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.112.124@KRBTEST.COM'} at 127.0.0.1:55252
I20260925 22:00:55.388459 15791 tablet_copy_service.cc:433] P 291a9916b523445ba24ef4216bb3cb60: ending tablet copy session a8eb490f50c1409db6576b612a354b5c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a8eb490f50c1409db6576b612a354b5c
I20260925 22:00:55.391515 16423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:55.392895 16528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.392942 16531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.397449 16529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.397696 16423 server_base.cc:1034] running on GCE node
I20260925 22:00:55.397805 16423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:55.397943 16423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:55.399120 16423 hybrid_clock.cc:648] HybridClock initialized: now 1790373655399107 us; error 31 us; skew 500 ppm
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:55.400475 16423 init.cc:377] Logged in from keytab as oryx/127.9.112.124@KRBTEST.COM (short username oryx)
I20260925 22:00:55.400807 16423 webserver.cc:460] Webserver started at http://127.9.112.124:32919/ using document root <none> and password file <none>
I20260925 22:00:55.400926 16423 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.400959 16423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.401554 16423 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:00:55.402050 16538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.402180 16423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.402220 16423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
uuid: "a8eb490f50c1409db6576b612a354b5c"
format_stamp: "Formatted at 2026-09-25 22:00:55 on dist-test-slave-794d"
server_key: "5a9b93e821f1b8f22ae92dac8447474c"
server_key_iv: "0c35a620a29a43b97e8f9eccb08c9fb0"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.402277 16423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.411520 16423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:55.413266 16423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:55.416126 16423 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.124:34087
I20260925 22:00:55.416471 16590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.124:34087 every 8 connection(s)
I20260925 22:00:55.416567 16423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
I20260925 22:00:55.421013 16591 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:00:55.421470 16591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Bootstrap starting.
I20260925 22:00:55.423847 16591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:00:55.424041 16591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Bootstrap complete.
I20260925 22:00:55.424229 16591 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } }
I20260925 22:00:55.424310 16591 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a8eb490f50c1409db6576b612a354b5c, State: Initialized, Role: LEARNER
I20260925 22:00:55.424373 16591 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } }
I20260925 22:00:55.424574 16591 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:55.424568 16594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.424785 16594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: This master's current role is: LEARNER
I20260925 22:00:55.425961 16605 catalog_manager.cc:1259] Loaded cluster ID: c845a698900342038899bdde37dfb7c6
I20260925 22:00:55.426039 16605 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: loading cluster ID for follower catalog manager: success
I20260925 22:00:55.427109 16605 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: acquiring CA information for follower catalog manager: success
I20260925 22:00:55.427390 16605 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 22:00:55.478557 16516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:55.478904 16516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:55.478950 16516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:55.480948 16516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:55.481027 16516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:55.481070 16516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:55.483532 16516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:42809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=34693
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:55.483820 16516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:55.484083 16516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:55.489516 16611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.489516 16613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.489522 16610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.489792 16516 server_base.cc:1034] running on GCE node
I20260925 22:00:55.490069 16516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:55.490394 16516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:55.491566 16516 hybrid_clock.cc:648] HybridClock initialized: now 1790373655491553 us; error 28 us; skew 500 ppm
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:55.495183 16516 init.cc:377] Logged in from keytab as oryx/127.9.112.65@KRBTEST.COM (short username oryx)
I20260925 22:00:55.495649 16516 webserver.cc:460] Webserver started at http://127.9.112.65:34693/ using document root <none> and password file <none>
I20260925 22:00:55.495841 16516 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.495899 16516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.497277 16516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:55.497905 16620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.498052 16516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:55.498135 16516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "2254d97d517b49adb1d96b2817bd98bc"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "b6894e5a641ef07f53100cd7710f9cd7"
server_key_iv: "ca8ad453f78c78baf4274108e7f4d998"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.498500 16516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.520668 16516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:55.522816 16516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:55.522966 16516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:55.523187 16516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:55.523480 16516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:55.523514 16516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.523543 16516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:55.523567 16516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.529881 16516 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:42809
I20260925 22:00:55.529949 16733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:42809 every 8 connection(s)
I20260925 22:00:55.530278 16516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.124@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
I20260925 22:00:55.539634 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 16516
I20260925 22:00:55.539801 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15967
I20260925 22:00:55.542452 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.531105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:34977 (local address 127.9.112.124:34087)
0925 22:00:55.531377 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.531378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.531492 (+ 114us) server_negotiation.cc:407] Connection header received
0925 22:00:55.532538 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.532539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.532550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.532616 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.533346 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.534090 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535508 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.535748 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535806 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.538589 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.538591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.538591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.538600 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.540936 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.541470 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.541471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.541472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.541531 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.541773 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.541774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.541774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.541897 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.541952 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.542255 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.542349 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":53,"threads_started":1}
I20260925 22:00:55.542661 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.531005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.9.112.65:34977)
0925 22:00:55.531425 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.531432 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.532275 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.532630 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.532631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.533047 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.533241 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.533247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.534206 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.534207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.535415 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.535418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.535820 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.535942 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.536591 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.536600 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.538463 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.541036 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.541050 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.541053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.541395 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.541589 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.541589 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.541589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.541716 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.541952 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.541959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.542127 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.542224 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.542408 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":352,"thread_start_us":54,"threads_started":1}
I20260925 22:00:55.543244 16734 heartbeater.cc:347] Connected to a master server at 127.9.112.124:34087
I20260925 22:00:55.543349 16734 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.543560 16734 heartbeater.cc:511] Master 127.9.112.124:34087 requested a full tablet report, sending...
I20260925 22:00:55.544104 16555 ts_manager.cc:194] Registered new tserver with Master: 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809)
I20260925 22:00:55.544376 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.531013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40793 (local address 127.9.112.126:43433)
0925 22:00:55.531304 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.531306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.531378 (+ 72us) server_negotiation.cc:407] Connection header received
0925 22:00:55.532437 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.532438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.532448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.532479 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.533339 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.534129 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535537 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.535820 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535872 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.539506 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.539508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.539508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.539517 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.543120 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.543481 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.543483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.543483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.543549 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.543731 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.543731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.543732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.543875 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.543938 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.544088 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.544234 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260925 22:00:55.544384 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.531005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:40793)
0925 22:00:55.531322 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.531330 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.532306 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.532562 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.532563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.533013 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.533241 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.533246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.534222 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.534223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.535422 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.535423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.535919 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.535968 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.536621 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.536626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.539364 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.543215 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.543216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.543217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.543421 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.543622 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.543623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.543623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.543681 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.543940 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.543941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.543997 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.544057 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.544234 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":258,"thread_start_us":86,"threads_started":1}
I20260925 22:00:55.544726 16735 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43433
I20260925 22:00:55.544770 16735 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.544895 16735 heartbeater.cc:511] Master 127.9.112.126:43433 requested a full tablet report, sending...
I20260925 22:00:55.545166 15771 ts_manager.cc:194] Re-registered known tserver with Master: 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809)
I20260925 22:00:55.545639 15771 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:40793
I20260925 22:00:55.545821 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:44873
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:43433
--webserver_port=46739
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:00:55.546738 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.531287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:43197 (local address 127.9.112.125:42739)
0925 22:00:55.531328 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.531330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.531340 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:55.532412 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.532414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.532426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.532489 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.533359 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.534129 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535430 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.535759 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.535820 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.539443 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.539445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.539445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.539455 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.542274 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.545932 (+ 3658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.545933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.545934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.546012 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.546211 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.546212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.546212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.546341 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.546398 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.546573 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.546649 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:55.546748 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.531004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.9.112.65:43197)
0925 22:00:55.531170 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.531216 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.532265 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.532549 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.532552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.533032 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.533241 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.533247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.534222 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.534223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.535343 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.535346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.536175 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.536176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.536233 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.536817 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.536821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.539319 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.545589 (+ 6270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.545590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.545591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.545818 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.546080 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.546081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.546082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.546136 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.546403 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.546404 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.546453 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.546516 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.546631 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":35,"thread_start_us":85,"threads_started":1}
I20260925 22:00:55.547097 16736 heartbeater.cc:347] Connected to a master server at 127.9.112.125:42739
I20260925 22:00:55.547142 16736 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.547220 16736 heartbeater.cc:511] Master 127.9.112.125:42739 requested a full tablet report, sending...
I20260925 22:00:55.547562 16369 ts_manager.cc:194] Registered new tserver with Master: 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809)
W20260925 22:00:55.630833 16742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:55.631055 16742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:55.631081 16742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:55.632280 16742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:55.632324 16742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:55.632376 16742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:55.633890 16742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:44873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=46739
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:55.634122 16742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:55.634336 16742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:55.639657 16750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.639581 16747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.639829 16742 server_base.cc:1034] running on GCE node
W20260925 22:00:55.639626 16748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.640089 16742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:55.640390 16742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:55.641562 16742 hybrid_clock.cc:648] HybridClock initialized: now 1790373655641516 us; error 61 us; skew 500 ppm
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:55.644284 16742 init.cc:377] Logged in from keytab as oryx/127.9.112.66@KRBTEST.COM (short username oryx)
I20260925 22:00:55.644791 16742 webserver.cc:460] Webserver started at http://127.9.112.66:46739/ using document root <none> and password file <none>
I20260925 22:00:55.645004 16742 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.645076 16742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.646519 16742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:55.647146 16757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.647387 16742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:55.647495 16742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "a083e388c8dd42708b9da73f667173fa"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "897cf09a753b633a436a04529800f925"
server_key_iv: "f8267b545d0fe93cbc1abecb921da687"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.647915 16742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.667873 16742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:55.670061 16742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:55.670204 16742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:55.670432 16742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:55.670729 16742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:55.670760 16742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.670789 16742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:55.670821 16742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.676796 16742 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:44873
I20260925 22:00:55.676846 16870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:44873 every 8 connection(s)
I20260925 22:00:55.677194 16742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:55.681496 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 16742
I20260925 22:00:55.681691 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 16101
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.124@KRBTEST.COM
I20260925 22:00:55.688921 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:43545
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:43433
--webserver_port=45283
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:00:55.689014 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.678089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:47217 (local address 127.9.112.125:42739)
0925 22:00:55.678166 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.678168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.678582 (+ 414us) server_negotiation.cc:407] Connection header received
0925 22:00:55.679061 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.679062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.679074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.679123 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.679817 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.680608 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.681933 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.682211 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.682251 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.685558 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.685560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.685560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.685570 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.687389 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687799 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.687800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.687800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.687852 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.688174 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.688174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.688175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.688317 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.688396 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.688822 (+ 426us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.688914 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 22:00:55.689014 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.678038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:46297 (local address 127.9.112.126:43433)
0925 22:00:55.678132 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.678134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.678424 (+ 290us) server_negotiation.cc:407] Connection header received
0925 22:00:55.679180 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.679181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.679192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.679241 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.679795 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.680292 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.681673 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.682024 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.682067 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.685558 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.685560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.685560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.685569 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.687172 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687779 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.687781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.687781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.687856 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.688174 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.688174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.688175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.688317 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.688395 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.688825 (+ 430us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.688914 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 22:00:55.689074 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.678161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:34121 (local address 127.9.112.124:34087)
0925 22:00:55.678199 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.678200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.678445 (+ 245us) server_negotiation.cc:407] Connection header received
0925 22:00:55.678984 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.678986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.679001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.679056 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.679783 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.680601 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.681679 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.682008 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.682062 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.686301 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.686302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.686303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.686312 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.687869 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.688291 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.688293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.688293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.688377 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.688553 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.688554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.688554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.688674 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.688715 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.688901 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.688991 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:55.689625 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.677930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.9.112.66:34121)
0925 22:00:55.678277 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.678324 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.678880 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.679132 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.679136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.679481 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.679684 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.679689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.680708 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.680709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.681573 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.681576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.682162 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.682162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.682308 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.683213 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.683218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.686170 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.687961 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.687963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.688193 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.688445 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.688446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.688446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.688498 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.688758 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.688759 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.688803 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.689179 (+ 376us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.689417 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":107,"thread_start_us":83,"threads_started":1}
I20260925 22:00:55.689625 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.677930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:46297)
0925 22:00:55.678293 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.678324 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.678891 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.679273 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.679274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.679579 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.679685 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.679693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.680427 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.680428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.681578 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.681581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.682159 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.682160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.682308 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.683107 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.683111 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.685428 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.687273 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687283 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.687287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.687691 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.687926 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.687927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.688097 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.688442 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.688449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.688661 (+ 212us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.689101 (+ 440us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.689338 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":20,"thread_start_us":82,"threads_started":1}
I20260925 22:00:55.690219 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.677930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.9.112.66:47217)
0925 22:00:55.678523 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.678531 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.678887 (+ 356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.679494 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.679494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.679674 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.679722 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.679724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.680677 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.680678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.681758 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.681761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.682320 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.682321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.682411 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.683055 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.683065 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.685428 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.687465 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687465 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.687466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.687708 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.687926 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.687927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.687927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.688097 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.688442 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.688449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.688661 (+ 212us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.690034 (+ 1373us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.690143 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":92,"thread_start_us":78,"threads_started":1}
I20260925 22:00:55.690635 16873 heartbeater.cc:347] Connected to a master server at 127.9.112.125:42739
I20260925 22:00:55.690766 16873 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.690972 16873 heartbeater.cc:511] Master 127.9.112.125:42739 requested a full tablet report, sending...
I20260925 22:00:55.691150 16871 heartbeater.cc:347] Connected to a master server at 127.9.112.124:34087
I20260925 22:00:55.691188 16871 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.691242 16871 heartbeater.cc:511] Master 127.9.112.124:34087 requested a full tablet report, sending...
I20260925 22:00:55.690671 16872 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43433
I20260925 22:00:55.691313 16872 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.691354 16872 heartbeater.cc:511] Master 127.9.112.126:43433 requested a full tablet report, sending...
I20260925 22:00:55.691443 16369 ts_manager.cc:194] Registered new tserver with Master: a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873)
I20260925 22:00:55.691531 16555 ts_manager.cc:194] Registered new tserver with Master: a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873)
I20260925 22:00:55.691686 15771 ts_manager.cc:194] Re-registered known tserver with Master: a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873)
I20260925 22:00:55.692209 15771 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:46297
W20260925 22:00:55.772578 16878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:55.772809 16878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:55.772832 16878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:55.774029 16878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:55.774067 16878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:55.774093 16878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:55.775619 16878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:43545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=45283
--tserver_master_addrs=127.9.112.126:43433,127.9.112.125:42739,127.9.112.124:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:55.775830 16878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:55.776023 16878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:55.780963 16884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.780990 16883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:55.781002 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:55.781203 16878 server_base.cc:1034] running on GCE node
I20260925 22:00:55.781404 16878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:55.781654 16878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:55.782848 16878 hybrid_clock.cc:648] HybridClock initialized: now 1790373655782832 us; error 31 us; skew 500 ppm
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:55.785485 16878 init.cc:377] Logged in from keytab as oryx/127.9.112.67@KRBTEST.COM (short username oryx)
I20260925 22:00:55.785956 16878 webserver.cc:460] Webserver started at http://127.9.112.67:45283/ using document root <none> and password file <none>
I20260925 22:00:55.786176 16878 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:55.786242 16878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:55.787750 16878 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:55.788478 16893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.788671 16878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:00:55.788770 16878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "36cedb52a1e74631bffaff0958d4dfa3"
format_stamp: "Formatted at 2026-09-25 22:00:54 on dist-test-slave-794d"
server_key: "4072bc4904ea6c827c973190451f4cb8"
server_key_iv: "1bc19e6e3ae9813d0fe80e0a45ccb602"
server_key_version: "encryptionkey@0"
I20260925 22:00:55.789137 16878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:55.802016 16878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:55.804235 16878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:55.804416 16878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:55.804661 16878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:00:55.805024 16878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:55.805060 16878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.805091 16878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:55.805109 16878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:55.810631 16878 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:43545
I20260925 22:00:55.810693 17006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:43545 every 8 connection(s)
I20260925 22:00:55.811013 16878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:55.815287 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 16878
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.125@KRBTEST.COM
Sep 25 22:00:55 dist-test-slave-794d krb5kdc[15726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.124@KRBTEST.COM
I20260925 22:00:55.821939 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.811948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:41811 (local address 127.9.112.125:42739)
0925 22:00:55.811996 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.811998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.812295 (+ 297us) server_negotiation.cc:407] Connection header received
0925 22:00:55.812666 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.812667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.812678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.812707 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.813563 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.814049 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815421 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.815740 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815773 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.818778 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.818779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.818780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.818787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.820454 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.820916 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.820918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.820918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.820973 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821220 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.821221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.821221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.821349 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.821422 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.821772 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.821845 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:00:55.822204 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.811747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:32895 (local address 127.9.112.124:34087)
0925 22:00:55.811792 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.811794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.812023 (+ 229us) server_negotiation.cc:407] Connection header received
0925 22:00:55.812665 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.812667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.812678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.812707 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.813568 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.814300 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.816851 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.817137 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.817182 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.819258 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.819259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.819259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.819266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.820910 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821233 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.821234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.821234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.821280 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821472 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.821473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.821473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.821583 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.821633 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.822051 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.822119 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:00:55.822315 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.811692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.9.112.67:41811)
0925 22:00:55.812251 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.812258 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.812562 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.812797 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.812798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.813261 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.813462 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.813469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.814140 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.814140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.815323 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.815325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.815851 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.816013 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.816684 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.816689 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.818702 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.820531 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.820538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.820540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.820837 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.821032 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821032 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.821033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.821152 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.821444 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.821451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.821612 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.822056 (+ 444us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.822135 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":215,"thread_start_us":49,"threads_started":1}
I20260925 22:00:55.822410 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.811692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.9.112.67:32895)
0925 22:00:55.811898 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.811943 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.812551 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.812787 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.812792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.813261 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.813462 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.813469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.815970 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.816754 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.816756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.817361 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.817361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.817428 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.817856 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.817861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.819161 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.820978 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.820979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.820979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.821152 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.821343 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.821344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.821413 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.821655 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.821656 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.821687 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.821763 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.821919 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":5,"thread_start_us":124,"threads_started":1}
I20260925 22:00:55.822649 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.811692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:41075)
0925 22:00:55.812143 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.812155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.812553 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.813058 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.813059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.813266 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:55.813460 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.813465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.814380 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.814380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.815323 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.815325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.815853 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.816017 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.816529 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:55.816539 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:55.818884 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:55.821348 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.821350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.821570 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.821990 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:55.821992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:55.822073 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:55.822346 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:55.822347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:55.822385 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.822444 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.822549 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":12,"thread_start_us":66,"threads_started":1}
I20260925 22:00:55.822727 17009 heartbeater.cc:347] Connected to a master server at 127.9.112.125:42739
I20260925 22:00:55.822813 17009 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.822976 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.812008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:41075 (local address 127.9.112.126:43433)
0925 22:00:55.812051 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.812053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.812218 (+ 165us) server_negotiation.cc:407] Connection header received
0925 22:00:55.812638 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.812639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.812651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.812705 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.813542 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.814304 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815424 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.815719 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.815758 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.818962 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.818963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.818964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.819216 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.821277 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.821638 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.821639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.821640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.821695 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.822131 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.822132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.822133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.822252 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.822324 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.822819 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.822893 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:00:55.822999 17009 heartbeater.cc:511] Master 127.9.112.125:42739 requested a full tablet report, sending...
I20260925 22:00:55.823246 17008 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43433
I20260925 22:00:55.823342 17007 heartbeater.cc:347] Connected to a master server at 127.9.112.124:34087
I20260925 22:00:55.823424 16369 ts_manager.cc:194] Registered new tserver with Master: 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545)
I20260925 22:00:55.823446 17008 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.823467 17007 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:55.823513 17008 heartbeater.cc:511] Master 127.9.112.126:43433 requested a full tablet report, sending...
I20260925 22:00:55.823531 17007 heartbeater.cc:511] Master 127.9.112.124:34087 requested a full tablet report, sending...
I20260925 22:00:55.823737 15771 ts_manager.cc:194] Re-registered known tserver with Master: 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545)
I20260925 22:00:55.823781 16555 ts_manager.cc:194] Registered new tserver with Master: 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545)
I20260925 22:00:55.824189 15771 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:41075
I20260925 22:00:55.826972 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.816381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.9.112.125:42739)
0925 22:00:55.818543 (+ 2162us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.818545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.818553 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:55.818574 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.818575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.818587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.819357 (+ 770us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.819394 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.820116 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.821166 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.821463 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.821748 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.824167 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.824169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.824169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.824175 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.825737 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.826020 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.826021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.826022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.826070 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.826241 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.826241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.826242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.826372 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.826470 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.826807 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.826883 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2141,"thread_start_us":57,"threads_started":1}
I20260925 22:00:55.829244 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.817255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.9.112.124:34087)
0925 22:00:55.817573 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.817574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.817580 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:00:55.817596 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.817596 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.817605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.817654 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.818394 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.819677 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.824158 (+ 4481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.824628 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.824665 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.826619 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.826620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.826621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.826626 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.828103 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.828464 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.828465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.828465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.828542 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.828724 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.828725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.828725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.828846 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.828938 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.829048 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.829144 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":44,"threads_started":1}
I20260925 22:00:55.830328 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.817878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.9.112.126:43433)
0925 22:00:55.820064 (+ 2186us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.820066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.820074 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:55.820092 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.820093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.820105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.824463 (+ 4358us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:55.824494 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.825186 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.825872 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.826147 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.826225 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.827614 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:55.827616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:55.827616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:55.827622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:55.828991 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.829287 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.829287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.829288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.829521 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:55.829843 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:55.829844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:55.829844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:55.829972 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:55.830057 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.830194 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.830259 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1856,"thread_start_us":86,"threads_started":1}
I20260925 22:00:55.831657 15771 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:55.833276 15771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:55.841027 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.837192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.9.112.67:43545)
0925 22:00:55.837393 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.837398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.837429 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:00:55.837471 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.837473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.837490 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.837949 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:55.838007 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.838656 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.839882 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.840659 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840750 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.840831 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.840901 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.840944 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260925 22:00:55.841022 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.837084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:43545 (local address 127.0.0.1:35552)
0925 22:00:55.837333 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.837342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.837383 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.837759 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.837760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.837763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:55.837815 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.837816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.838742 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.838743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.839794 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.839796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.840759 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.840820 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.840825 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.840884 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.840928 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":83,"threads_started":1}
I20260925 22:00:55.841192 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.837346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48002 (local address 127.9.112.65:42809)
0925 22:00:55.837645 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.837648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.837656 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:55.837688 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.837689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.837702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.837772 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:55.837965 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.839002 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840015 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.840692 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840752 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.840831 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.841057 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.841099 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":75,"threads_started":1}
I20260925 22:00:55.841171 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.837063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:44873 (local address 127.0.0.1:56074)
0925 22:00:55.837215 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.837226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.837265 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.837672 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.837673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.837674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:55.837740 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.837742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.839078 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.839079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.840108 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.840110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.840983 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.841042 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.841046 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.841073 (+ 27us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.841113 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":33}
I20260925 22:00:55.841405 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.837158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42809 (local address 127.0.0.1:48002)
0925 22:00:55.837499 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.837508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.837547 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.837822 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.837823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.837825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:55.837873 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.837874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.839088 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.839089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.839933 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.839936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.840859 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.840912 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.840916 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.840955 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.840987 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":2,"thread_start_us":122,"threads_started":1}
I20260925 22:00:55.841924 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.837355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.9.112.66:44873)
0925 22:00:55.837507 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.837512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.837523 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:00:55.837559 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.837561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.837578 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.837806 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:55.837858 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.838991 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.840187 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.840895 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.841381 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.841804 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.841828 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.841865 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260925 22:00:55.842056 16941 tablet_service.cc:1467] Processing CreateTablet for tablet 42bf46958e344b27a6e14b257ab8f292 (DEFAULT_TABLE table=test-table [id=8172bd06ea0e4657852c765f6b02c8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:55.842056 16668 tablet_service.cc:1467] Processing CreateTablet for tablet 42bf46958e344b27a6e14b257ab8f292 (DEFAULT_TABLE table=test-table [id=8172bd06ea0e4657852c765f6b02c8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:55.842423 16941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42bf46958e344b27a6e14b257ab8f292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:55.842473 16668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42bf46958e344b27a6e14b257ab8f292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:55.842480 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 42bf46958e344b27a6e14b257ab8f292 (DEFAULT_TABLE table=test-table [id=8172bd06ea0e4657852c765f6b02c8c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:55.842782 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42bf46958e344b27a6e14b257ab8f292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:55.845494 17033 tablet_bootstrap.cc:492] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Bootstrap starting.
I20260925 22:00:55.846109 17034 tablet_bootstrap.cc:492] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Bootstrap starting.
I20260925 22:00:55.846225 17033 tablet_bootstrap.cc:654] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:55.846586 17033 log.cc:824] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:55.846735 17034 tablet_bootstrap.cc:654] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:55.847000 17034 log.cc:824] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:55.847303 17035 tablet_bootstrap.cc:492] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Bootstrap starting.
I20260925 22:00:55.847411 17033 tablet_bootstrap.cc:492] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: No bootstrap required, opened a new log
I20260925 22:00:55.847481 17033 ts_tablet_manager.cc:1397] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:55.847658 17034 tablet_bootstrap.cc:492] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: No bootstrap required, opened a new log
I20260925 22:00:55.847715 17034 ts_tablet_manager.cc:1397] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:55.849169 17035 tablet_bootstrap.cc:654] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:55.849336 17033 raft_consensus.cc:348] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:55.849514 17033 raft_consensus.cc:374] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:55.849558 17033 raft_consensus.cc:729] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a083e388c8dd42708b9da73f667173fa, State: Initialized, Role: FOLLOWER
I20260925 22:00:55.849525 17035 log.cc:824] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:55.849678 17033 consensus_queue.cc:260] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:55.849941 17033 ts_tablet_manager.cc:1428] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:00:55.850016 16872 heartbeater.cc:503] Master 127.9.112.126:43433 was elected leader, sending a full tablet report...
I20260925 22:00:55.850163 17034 raft_consensus.cc:348] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:55.850430 17034 raft_consensus.cc:374] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:55.850481 17034 raft_consensus.cc:729] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36cedb52a1e74631bffaff0958d4dfa3, State: Initialized, Role: FOLLOWER
I20260925 22:00:55.850603 17034 consensus_queue.cc:260] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:55.850946 17034 ts_tablet_manager.cc:1428] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:55.851377 17008 heartbeater.cc:503] Master 127.9.112.126:43433 was elected leader, sending a full tablet report...
I20260925 22:00:55.852276 17035 tablet_bootstrap.cc:492] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: No bootstrap required, opened a new log
I20260925 22:00:55.852371 17035 ts_tablet_manager.cc:1397] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 22:00:55.854208 17035 raft_consensus.cc:348] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:55.854354 17035 raft_consensus.cc:374] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:55.854384 17035 raft_consensus.cc:729] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2254d97d517b49adb1d96b2817bd98bc, State: Initialized, Role: FOLLOWER
I20260925 22:00:55.854501 17035 consensus_queue.cc:260] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:55.854743 17035 ts_tablet_manager.cc:1428] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:55.854846 16735 heartbeater.cc:503] Master 127.9.112.126:43433 was elected leader, sending a full tablet report...
W20260925 22:00:55.928153 16874 tablet.cc:2369] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:55.968788 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.965931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38062 (local address 127.9.112.124:34087)
0925 22:00:55.966017 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:55.966019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:55.966029 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:55.966057 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:55.966058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:55.966113 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:55.966181 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:55.966428 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.966943 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.967959 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.968471 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.968544 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.968609 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:55.968688 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 22:00:55.968725 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 22:00:55.968835 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:55.965815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.0.0.1:38062)
0925 22:00:55.965907 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:55.965922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:55.965987 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:55.966255 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:55.966256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:55.966258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:55.966307 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.966309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.967038 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.967039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.967853 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:55.967856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:55.968570 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:55.968571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:55.968632 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:55.968636 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:55.968680 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:00:55.968770 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 22:00:55.969249 16562 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260925 22:00:55.970124 17042 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291a9916b523445ba24ef4216bb3cb60. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:55.970221 17042 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: This master's current role is: LEARNER
W20260925 22:00:56.031941 16737 tablet.cc:2369] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:00:56.062618 17010 tablet.cc:2369] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:56.175072 17040 raft_consensus.cc:482] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:56.175200 17040 raft_consensus.cc:504] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:56.175519 17040 leader_election.cc:288] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873), 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809)
I20260925 22:00:56.179725 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.175904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:32981 (local address 127.9.112.65:42809)
0925 22:00:56.175942 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:56.175944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:56.175950 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:00:56.175972 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:56.175973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:56.175981 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:56.176038 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:56.176295 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.177098 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.178698 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.179279 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.179350 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.179357 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:56.179608 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 22:00:56.179654 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:56.179742 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.175730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:44873 (local address 127.9.112.67:50185)
0925 22:00:56.175779 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:56.175792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:56.175850 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:56.176079 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:56.176080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:56.176083 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:56.176199 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:56.176201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.177205 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.177206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:56.178578 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:56.178582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.179419 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.179420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:56.179510 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.179517 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 22:00:56.179586 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:00:56.179639 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":89}
I20260925 22:00:56.179764 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.175859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:50185 (local address 127.9.112.66:44873)
0925 22:00:56.175904 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:56.175906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:56.175917 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:00:56.175947 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:56.175947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:56.175958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:56.176038 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:56.176295 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.177098 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.178728 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.179262 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.179347 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.179357 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:56.179608 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 22:00:56.179708 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:00:56.179762 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.175830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42809 (local address 127.9.112.67:32981)
0925 22:00:56.175872 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:56.175880 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:56.175905 (+ 25us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:56.176079 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:56.176080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:56.176083 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:56.176202 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:56.176203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.177207 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.177207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:56.178578 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:56.178582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.179419 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.179420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:56.179511 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.179517 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:56.179586 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:00:56.179639 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":71}
I20260925 22:00:56.180188 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42bf46958e344b27a6e14b257ab8f292" candidate_uuid: "36cedb52a1e74631bffaff0958d4dfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a083e388c8dd42708b9da73f667173fa" is_pre_election: true
I20260925 22:00:56.180199 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42bf46958e344b27a6e14b257ab8f292" candidate_uuid: "36cedb52a1e74631bffaff0958d4dfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2254d97d517b49adb1d96b2817bd98bc" is_pre_election: true
I20260925 22:00:56.180382 16825 raft_consensus.cc:2454] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36cedb52a1e74631bffaff0958d4dfa3 in term 0.
I20260925 22:00:56.180485 16688 raft_consensus.cc:2454] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36cedb52a1e74631bffaff0958d4dfa3 in term 0.
I20260925 22:00:56.180619 16897 leader_election.cc:302] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cedb52a1e74631bffaff0958d4dfa3, a083e388c8dd42708b9da73f667173fa; no voters:
I20260925 22:00:56.180778 17040 raft_consensus.cc:2781] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:56.180845 17040 raft_consensus.cc:482] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:56.180867 17040 raft_consensus.cc:3037] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:56.181598 17040 raft_consensus.cc:504] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:56.181727 17040 leader_election.cc:288] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [CANDIDATE]: Term 1 election: Requested vote from peers a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873), 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809)
I20260925 22:00:56.181974 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42bf46958e344b27a6e14b257ab8f292" candidate_uuid: "36cedb52a1e74631bffaff0958d4dfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a083e388c8dd42708b9da73f667173fa"
I20260925 22:00:56.181967 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42bf46958e344b27a6e14b257ab8f292" candidate_uuid: "36cedb52a1e74631bffaff0958d4dfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2254d97d517b49adb1d96b2817bd98bc"
I20260925 22:00:56.182052 16825 raft_consensus.cc:3037] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:56.182057 16688 raft_consensus.cc:3037] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:56.182699 16688 raft_consensus.cc:2454] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36cedb52a1e74631bffaff0958d4dfa3 in term 1.
I20260925 22:00:56.182773 16825 raft_consensus.cc:2454] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36cedb52a1e74631bffaff0958d4dfa3 in term 1.
I20260925 22:00:56.182914 16897 leader_election.cc:302] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cedb52a1e74631bffaff0958d4dfa3, a083e388c8dd42708b9da73f667173fa; no voters:
I20260925 22:00:56.183000 17040 raft_consensus.cc:2781] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:56.183156 17040 raft_consensus.cc:686] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 LEADER]: Becoming Leader. State: Replica: 36cedb52a1e74631bffaff0958d4dfa3, State: Running, Role: LEADER
I20260925 22:00:56.183266 17040 consensus_queue.cc:237] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } }
I20260925 22:00:56.184255 15771 catalog_manager.cc:5495] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67). New cstate: current_term: 1 leader_uuid: "36cedb52a1e74631bffaff0958d4dfa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:56.184832 16519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: NON_VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: true } } }
I20260925 22:00:56.184942 16519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:56.185568 17049 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: attempting to promote NON_VOTER a8eb490f50c1409db6576b612a354b5c to VOTER
I20260925 22:00:56.185689 17049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } }
I20260925 22:00:56.186051 16379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Refusing update from remote peer 291a9916b523445ba24ef4216bb3cb60: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 22:00:56.186132 16562 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 LEARNER]: Refusing update from remote peer 291a9916b523445ba24ef4216bb3cb60: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 22:00:56.186235 16519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 22:00:56.186357 16519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 22:00:56.186949 17049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a8eb490f50c1409db6576b612a354b5c (127.9.112.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.187079 16379 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a8eb490f50c1409db6576b612a354b5c (127.9.112.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.187911 17049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.187988 17049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:56.187958 16331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.188045 16331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:56.188035 17049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.188073 17049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60 [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:56.188131 16562 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a8eb490f50c1409db6576b612a354b5c (127.9.112.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.188433 17051 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.188525 17051 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:00:56.189037 17042 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "291a9916b523445ba24ef4216bb3cb60" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:56.189136 17042 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:00:56.198571 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.196119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.9.112.67:43545)
0925 22:00:56.196177 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:56.196179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:56.196189 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:56.196324 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:56.196326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:56.196370 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:56.196433 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:56.196570 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.197124 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.197857 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.198044 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.198099 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.198266 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:56.198375 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:56.198449 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:56.198467 (+ 18us) server_negotiation.cc:299] Negotiation successful
0925 22:00:56.198490 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:00:56.200510 16688 raft_consensus.cc:1261] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 1 FOLLOWER]: Refusing update from remote peer 36cedb52a1e74631bffaff0958d4dfa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:56.200536 16825 raft_consensus.cc:1261] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Refusing update from remote peer 36cedb52a1e74631bffaff0958d4dfa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:56.200872 17040 consensus_queue.cc:1035] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:56.201025 17047 consensus_queue.cc:1035] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:56.205003 17055 mvcc.cc:205] Tried to move back new op lower bound from 7333370495794327552 to 7333370495727169536. Current Snapshot: MvccSnapshot[applied={T|T < 7333370495794327552}]
I20260925 22:00:56.205629 17054 mvcc.cc:205] Tried to move back new op lower bound from 7333370495794327552 to 7333370495727169536. Current Snapshot: MvccSnapshot[applied={T|T < 7333370495794327552}]
I20260925 22:00:56.206467 15769 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:56.206552 15769 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:56.207777 15769 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 291a9916b523445ba24ef4216bb3cb60: Sending DeleteTablet for 3 replicas of tablet 42bf46958e344b27a6e14b257ab8f292
I20260925 22:00:56.208153 16805 tablet_service.cc:1514] Processing DeleteTablet for tablet 42bf46958e344b27a6e14b257ab8f292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:56 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:56074
I20260925 22:00:56.208153 16941 tablet_service.cc:1514] Processing DeleteTablet for tablet 42bf46958e344b27a6e14b257ab8f292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:56 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:35552
I20260925 22:00:56.208171 16668 tablet_service.cc:1514] Processing DeleteTablet for tablet 42bf46958e344b27a6e14b257ab8f292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:56 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:48002
I20260925 22:00:56.208737 17065 tablet_replica.cc:330] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: stopping tablet replica
I20260925 22:00:56.208869 17063 tablet_replica.cc:330] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: stopping tablet replica
I20260925 22:00:56.209103 17063 raft_consensus.cc:2229] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:56.209342 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 15739
I20260925 22:00:56.209262 17065 raft_consensus.cc:2229] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:56.215615 17065 raft_consensus.cc:2258] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:56.215930 17063 raft_consensus.cc:2258] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:56.216728 17065 ts_tablet_manager.cc:1905] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:56.217547 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.208952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.9.112.124:34087)
0925 22:00:56.209008 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:56.209011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:56.209128 (+ 117us) server_negotiation.cc:407] Connection header received
0925 22:00:56.209220 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:56.209221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:56.209242 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:56.209271 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:56.209988 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.210756 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.211840 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.212157 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.212201 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.214366 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:56.214367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:56.214368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:56.214376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:56.216187 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:56.216647 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:56.216648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:56.216648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:56.216712 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:56.216918 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:56.216919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:56.216920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:56.217053 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:56.217130 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:56.217358 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 22:00:56.217447 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:00:56.218250 17063 ts_tablet_manager.cc:1905] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:56.219780 17063 ts_tablet_manager.cc:1918] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:56.219851 17063 log.cc:1196] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/42bf46958e344b27a6e14b257ab8f292
I20260925 22:00:56.220084 17063 ts_tablet_manager.cc:1939] T 42bf46958e344b27a6e14b257ab8f292 P a083e388c8dd42708b9da73f667173fa: Deleting consensus metadata
I20260925 22:00:56.220585 17065 ts_tablet_manager.cc:1918] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:56.220669 17065 log.cc:1196] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/42bf46958e344b27a6e14b257ab8f292
I20260925 22:00:56.220908 17065 ts_tablet_manager.cc:1939] T 42bf46958e344b27a6e14b257ab8f292 P 36cedb52a1e74631bffaff0958d4dfa3: Deleting consensus metadata
W20260925 22:00:56.221307 16760 connection.cc:441] server connection from 127.0.0.1:56074 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56074 (request call id 1) could send its response
W20260925 22:00:56.221630 16896 connection.cc:441] server connection from 127.0.0.1:35552 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35552 (request call id 1) could send its response
I20260925 22:00:56.221836 17064 tablet_replica.cc:330] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: stopping tablet replica
I20260925 22:00:56.221978 17064 raft_consensus.cc:2229] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:56.222100 17064 raft_consensus.cc:2258] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:56.222770 17064 ts_tablet_manager.cc:1905] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:56.224449 17064 ts_tablet_manager.cc:1918] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:56.224517 17064 log.cc:1196] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/42bf46958e344b27a6e14b257ab8f292
I20260925 22:00:56.224761 17064 ts_tablet_manager.cc:1939] T 42bf46958e344b27a6e14b257ab8f292 P 2254d97d517b49adb1d96b2817bd98bc: Deleting consensus metadata
W20260925 22:00:56.227021 16624 connection.cc:441] server connection from 127.0.0.1:48002 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48002 (request call id 1) could send its response
I20260925 22:00:56.227345 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.218501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.9.112.124:34087)
0925 22:00:56.218534 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:56.218536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:56.218894 (+ 358us) server_negotiation.cc:407] Connection header received
0925 22:00:56.218982 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:56.218983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:56.218992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:56.219019 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:56.219803 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.220616 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.221752 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.222054 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.222092 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.224099 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:56.224100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:56.224101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:56.224108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:56.226182 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:56.226541 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:56.226543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:56.226543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:56.226597 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:56.226783 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:56.226784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:56.226784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:56.226901 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:56.226987 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:56.227121 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 22:00:56.227205 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 22:00:56.228186 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:56.218324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.9.112.125:42739)
0925 22:00:56.218371 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:56.218373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:56.218864 (+ 491us) server_negotiation.cc:407] Connection header received
0925 22:00:56.218957 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:56.218958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:56.218973 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:56.219001 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:56.219735 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.220619 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.221987 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:56.222292 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:56.222329 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:56.223937 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:56.223938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:56.223939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:56.223947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:56.226680 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:56.227301 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:56.227302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:56.227303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:56.227367 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:56.227581 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:56.227583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:56.227583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:56.227729 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:56.227827 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:56.228033 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 22:00:56.228125 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:00:57.205353 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.205014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:59099)
0925 22:00:57.205177 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.205295 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
W20260925 22:00:57.205775 16735 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:43433 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.206020 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.205543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:59145)
0925 22:00:57.205906 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.205976 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":48,"threads_started":1}
I20260925 22:00:57.206163 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.205770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:58113)
0925 22:00:57.206000 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.206065 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":65,"threads_started":1}
W20260925 22:00:57.206202 16872 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:43433 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
W20260925 22:00:57.206322 17008 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:43433 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.708074 17077 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 291a9916b523445ba24ef4216bb3cb60)
I20260925 22:00:57.708413 17077 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } }
I20260925 22:00:57.708865 17077 leader_election.cc:288] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433), d06122da75b64a4da086ec98076cafb9 (127.9.112.125:42739)
I20260925 22:00:57.708906 17078 raft_consensus.cc:482] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 291a9916b523445ba24ef4216bb3cb60)
I20260925 22:00:57.709082 17078 raft_consensus.cc:504] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } }
I20260925 22:00:57.709343 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.709081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:52948)
0925 22:00:57.709234 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.709294 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":60,"threads_started":1}
I20260925 22:00:57.709452 17078 leader_election.cc:288] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433), a8eb490f50c1409db6576b612a354b5c (127.9.112.124:34087)
W20260925 22:00:57.709496 16540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.710214 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.709848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:52958)
0925 22:00:57.710153 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.710185 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286}
W20260925 22:00:57.710315 16540 leader_election.cc:334] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433): Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.710494 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.709588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:52956)
0925 22:00:57.710369 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.710447 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":82,"threads_started":1}
W20260925 22:00:57.710642 16354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.711100 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.711013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:52986)
0925 22:00:57.711051 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.711074 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 22:00:57.711179 16354 leader_election.cc:334] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433): Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.712823 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.709107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:42739 (local address 127.0.0.1:52280)
0925 22:00:57.709351 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.709362 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.709471 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.709730 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.709731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.709733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.709805 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.709808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.710776 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.710777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.711852 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.711856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.712544 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.712545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.712617 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.712623 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.712693 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.712746 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260925 22:00:57.712832 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.709187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.9.112.125:42739)
0925 22:00:57.709300 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.709302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.709439 (+ 137us) server_negotiation.cc:407] Connection header received
0925 22:00:57.709576 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.709578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.709593 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.709641 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.709892 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.710682 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.711948 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.712458 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.712525 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.712537 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.712701 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.712733 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260925 22:00:57.713146 16379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8eb490f50c1409db6576b612a354b5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d06122da75b64a4da086ec98076cafb9" is_pre_election: true
I20260925 22:00:57.713264 16379 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8eb490f50c1409db6576b612a354b5c in term 1.
I20260925 22:00:57.713445 16541 leader_election.cc:302] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a8eb490f50c1409db6576b612a354b5c, d06122da75b64a4da086ec98076cafb9; no voters: 291a9916b523445ba24ef4216bb3cb60
I20260925 22:00:57.713605 17077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 22:00:57.713646 17077 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 291a9916b523445ba24ef4216bb3cb60)
I20260925 22:00:57.713677 17077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:00:57.713920 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.709634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:34087 (local address 127.0.0.1:38082)
0925 22:00:57.710470 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.710482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.710556 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.710856 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.710857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.710859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.710911 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.710913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.711784 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.711785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.712803 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.712807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.713621 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.713622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.713725 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.713731 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.713791 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.713847 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":34,"threads_started":1}
I20260925 22:00:57.713990 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.710108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.9.112.124:34087)
0925 22:00:57.710438 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.710441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.710644 (+ 203us) server_negotiation.cc:407] Connection header received
0925 22:00:57.710678 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.710680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.710692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.710785 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.711032 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.711696 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.712902 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.713528 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.713621 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.713636 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.713844 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.713907 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":91,"threads_started":1}
I20260925 22:00:57.714218 16562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d06122da75b64a4da086ec98076cafb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a8eb490f50c1409db6576b612a354b5c" is_pre_election: true
I20260925 22:00:57.714762 17077 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } }
I20260925 22:00:57.714926 16562 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d06122da75b64a4da086ec98076cafb9 in current term 2: Already voted for candidate a8eb490f50c1409db6576b612a354b5c in this term.
I20260925 22:00:57.714936 17077 leader_election.cc:288] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [CANDIDATE]: Term 2 election: Requested vote from peers 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433), d06122da75b64a4da086ec98076cafb9 (127.9.112.125:42739)
I20260925 22:00:57.715207 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.715052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:53000)
0925 22:00:57.715156 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.715186 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 22:00:57.715202 16356 leader_election.cc:302] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d06122da75b64a4da086ec98076cafb9; no voters: 291a9916b523445ba24ef4216bb3cb60, a8eb490f50c1409db6576b612a354b5c
I20260925 22:00:57.715153 16379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8eb490f50c1409db6576b612a354b5c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d06122da75b64a4da086ec98076cafb9"
I20260925 22:00:57.715274 16379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:00:57.715910 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.715694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:53014)
0925 22:00:57.715852 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.715889 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260925 22:00:57.716148 16540 leader_election.cc:334] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433): Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
I20260925 22:00:57.716190 16379 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8eb490f50c1409db6576b612a354b5c in term 2.
I20260925 22:00:57.716511 17078 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:00:57.716594 16541 leader_election.cc:302] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a8eb490f50c1409db6576b612a354b5c, d06122da75b64a4da086ec98076cafb9; no voters: 291a9916b523445ba24ef4216bb3cb60
I20260925 22:00:57.716717 17077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:00:57.716912 17077 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [term 2 LEADER]: Becoming Leader. State: Replica: a8eb490f50c1409db6576b612a354b5c, State: Running, Role: LEADER
I20260925 22:00:57.717058 17077 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } }
I20260925 22:00:57.717558 17087 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8eb490f50c1409db6576b612a354b5c. Latest consensus state: current_term: 2 leader_uuid: "a8eb490f50c1409db6576b612a354b5c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:57.717707 17087 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:57.718007 17089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:57.718521 17089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:00:57.718592 17089 catalog_manager.cc:1259] Loaded cluster ID: c845a698900342038899bdde37dfb7c6
I20260925 22:00:57.718624 17089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:57.718811 17089 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:57.718883 17089 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Loaded TSK: 0
I20260925 22:00:57.719036 17089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:00:57.771210 16554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:57.773643 16554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:57.775156 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.774968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:53032)
0925 22:00:57.775036 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.775120 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 22:00:57.775195 16379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [term 2 FOLLOWER]: Refusing update from remote peer a8eb490f50c1409db6576b612a354b5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260925 22:00:57.775636 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.775474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.0.0.1:53036)
0925 22:00:57.775529 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.775561 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:00:57.775569 17087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 22:00:57.775782 16540 consensus_peers.cc:487] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c -> Peer 291a9916b523445ba24ef4216bb3cb60 (127.9.112.126:43433): Couldn't send request to peer 291a9916b523445ba24ef4216bb3cb60. Status: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 22:00:57.776477 17078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8eb490f50c1409db6576b612a354b5c. Latest consensus state: current_term: 2 leader_uuid: "a8eb490f50c1409db6576b612a354b5c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:57.776652 17078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:00:57.777117 17077 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a8eb490f50c1409db6576b612a354b5c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:57.777226 17090 mvcc.cc:205] Tried to move back new op lower bound from 7333370502244040704 to 7333370502009790464. Current Snapshot: MvccSnapshot[applied={T|T < 7333370502244040704 or (T in {7333370502244040704})}]
I20260925 22:00:57.777240 17077 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:57.777701 17078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a8eb490f50c1409db6576b612a354b5c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "291a9916b523445ba24ef4216bb3cb60" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43433 } } peers { permanent_uuid: "d06122da75b64a4da086ec98076cafb9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 42739 } attrs { promote: false } } peers { permanent_uuid: "a8eb490f50c1409db6576b612a354b5c" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 34087 } attrs { promote: false } } }
I20260925 22:00:57.777814 17078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06122da75b64a4da086ec98076cafb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:00:57.777910 17091 mvcc.cc:205] Tried to move back new op lower bound from 7333370502244040704 to 7333370502009790464. Current Snapshot: MvccSnapshot[applied={T|T < 7333370502244040704 or (T in {7333370502244040704})}]
I20260925 22:00:57.783417 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.780374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.9.112.65:42809)
0925 22:00:57.780514 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.780755 (+ 241us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.780764 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:57.780794 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.780795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.780806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.780874 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.780981 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.781548 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.782604 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.783055 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.783151 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.783171 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.783271 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.783318 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260925 22:00:57.783416 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.779875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42809 (local address 127.0.0.1:48018)
0925 22:00:57.780142 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.780152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.780186 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.780888 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.780889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.780890 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.780931 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.780932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.781625 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.781626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.782508 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.782511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.783134 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.783135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.783202 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.783208 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.783262 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.783318 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":27,"thread_start_us":126,"threads_started":1}
I20260925 22:00:57.783883 16668 tablet_service.cc:1467] Processing CreateTablet for tablet 8edb6064d04842e5a975c7cdecb0d292 (DEFAULT_TABLE table=test-table [id=de3dec0317f641b3a00e5159282c475e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:57.783921 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.779714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:44873 (local address 127.0.0.1:56078)
0925 22:00:57.779831 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.779842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.779894 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.780632 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.780633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.780637 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.780719 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.780721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.781704 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.781705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.783026 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.783030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.783693 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.783694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.783760 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.783766 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.783795 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.783850 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":6}
I20260925 22:00:57.784163 16668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8edb6064d04842e5a975c7cdecb0d292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:57.784238 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.780177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.9.112.66:44873)
0925 22:00:57.780420 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.780422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.780430 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:57.780514 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.780516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.780528 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.780619 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.780795 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.781595 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.783154 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.783599 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.784091 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.784111 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.784129 (+ 18us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.784173 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":62,"threads_started":1}
I20260925 22:00:57.784528 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 8edb6064d04842e5a975c7cdecb0d292 (DEFAULT_TABLE table=test-table [id=de3dec0317f641b3a00e5159282c475e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:57.784678 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8edb6064d04842e5a975c7cdecb0d292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:57.785552 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.779714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:43545 (local address 127.0.0.1:35568)
0925 22:00:57.780127 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.780137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.780505 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.780725 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.780727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.780730 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.780798 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.780800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.781807 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.781808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.783026 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.783030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.785263 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.785264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.785335 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.785341 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.785411 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.785465 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":46}
I20260925 22:00:57.785797 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.779816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.9.112.67:43545)
0925 22:00:57.779968 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.779970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.780266 (+ 296us) server_negotiation.cc:407] Connection header received
0925 22:00:57.780609 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.780611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.780625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.780717 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.780928 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.781703 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.783714 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.785136 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.785225 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.785244 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.785440 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.785507 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260925 22:00:57.786049 17100 tablet_bootstrap.cc:492] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Bootstrap starting.
I20260925 22:00:57.786345 16941 tablet_service.cc:1467] Processing CreateTablet for tablet 8edb6064d04842e5a975c7cdecb0d292 (DEFAULT_TABLE table=test-table [id=de3dec0317f641b3a00e5159282c475e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:57.786420 17100 tablet_bootstrap.cc:654] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:57.786423 17101 tablet_bootstrap.cc:492] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc: Bootstrap starting.
I20260925 22:00:57.786494 16941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8edb6064d04842e5a975c7cdecb0d292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:57.786872 17101 tablet_bootstrap.cc:654] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:57.786993 17100 tablet_bootstrap.cc:492] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: No bootstrap required, opened a new log
I20260925 22:00:57.787051 17100 ts_tablet_manager.cc:1397] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:57.787211 17100 raft_consensus.cc:348] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.787295 17100 raft_consensus.cc:374] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:57.787317 17100 raft_consensus.cc:729] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a083e388c8dd42708b9da73f667173fa, State: Initialized, Role: FOLLOWER
I20260925 22:00:57.787381 17100 consensus_queue.cc:260] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.787537 17100 ts_tablet_manager.cc:1428] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:57.787561 17101 tablet_bootstrap.cc:492] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc: No bootstrap required, opened a new log
I20260925 22:00:57.787626 17101 ts_tablet_manager.cc:1397] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:57.787788 17101 raft_consensus.cc:348] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.787879 17101 raft_consensus.cc:374] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:57.787897 17101 raft_consensus.cc:729] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2254d97d517b49adb1d96b2817bd98bc, State: Initialized, Role: FOLLOWER
I20260925 22:00:57.787957 17101 consensus_queue.cc:260] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.788106 17101 ts_tablet_manager.cc:1428] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:57.788296 16871 heartbeater.cc:503] Master 127.9.112.124:34087 was elected leader, sending a full tablet report...
I20260925 22:00:57.788650 17106 tablet_bootstrap.cc:492] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Bootstrap starting.
I20260925 22:00:57.788712 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.788395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:40963)
0925 22:00:57.788567 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.788661 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260925 22:00:57.789041 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.788965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:38609)
0925 22:00:57.788996 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.789018 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 22:00:57.789074 17106 tablet_bootstrap.cc:654] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:57.789752 17106 tablet_bootstrap.cc:492] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: No bootstrap required, opened a new log
I20260925 22:00:57.789800 17106 ts_tablet_manager.cc:1397] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:57.789932 17106 raft_consensus.cc:348] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.790007 17106 raft_consensus.cc:374] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:57.790026 17106 raft_consensus.cc:729] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36cedb52a1e74631bffaff0958d4dfa3, State: Initialized, Role: FOLLOWER
I20260925 22:00:57.790102 17106 consensus_queue.cc:260] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.790270 17106 ts_tablet_manager.cc:1428] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:57.790320 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.788685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:47863)
0925 22:00:57.790075 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.790258 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1302,"thread_start_us":105,"threads_started":1}
I20260925 22:00:57.790830 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.790563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:59007)
0925 22:00:57.790717 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.790766 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260925 22:00:57.791047 17007 heartbeater.cc:503] Master 127.9.112.124:34087 was elected leader, sending a full tablet report...
I20260925 22:00:57.791215 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.790972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:49345)
0925 22:00:57.791150 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.791180 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260925 22:00:57.791433 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.791338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:38841)
0925 22:00:57.791381 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.791404 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 22:00:57.865008 17104 raft_consensus.cc:482] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:57.865119 17104 raft_consensus.cc:504] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.865502 17104 leader_election.cc:288] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809), 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545)
I20260925 22:00:57.868929 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.865849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:55703 (local address 127.9.112.65:42809)
0925 22:00:57.865899 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.865902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.866045 (+ 143us) server_negotiation.cc:407] Connection header received
0925 22:00:57.866129 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.866130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.866140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.866205 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.866350 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.866899 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.868109 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.868561 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.868625 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.868637 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.868796 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.868847 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:00:57.868993 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.865870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:53183 (local address 127.9.112.67:43545)
0925 22:00:57.865940 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.865942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.865954 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:00:57.866044 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.866046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.866057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.866156 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:57.866363 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.866890 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.868160 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.868593 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.868671 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.868686 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.868873 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.868927 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:00:57.869083 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.865710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:43545 (local address 127.9.112.66:53183)
0925 22:00:57.865865 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.865878 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.865945 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.866183 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.866185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.866187 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.866272 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.866275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.866977 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.866979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.868006 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.868009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.868711 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.868712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.868784 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.868792 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.868854 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.868911 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":56}
I20260925 22:00:57.868919 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.865775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42809 (local address 127.9.112.66:55703)
0925 22:00:57.865873 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.865888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:57.866051 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:57.866216 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:57.866217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:57.866219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:57.866280 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.866282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.867045 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.867047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.867987 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:57.867990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.868649 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.868650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:57.868719 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.868723 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:57.868770 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 22:00:57.868819 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 22:00:57.869367 16961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edb6064d04842e5a975c7cdecb0d292" candidate_uuid: "a083e388c8dd42708b9da73f667173fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cedb52a1e74631bffaff0958d4dfa3" is_pre_election: true
I20260925 22:00:57.869447 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edb6064d04842e5a975c7cdecb0d292" candidate_uuid: "a083e388c8dd42708b9da73f667173fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2254d97d517b49adb1d96b2817bd98bc" is_pre_election: true
I20260925 22:00:57.869540 16688 raft_consensus.cc:2454] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a083e388c8dd42708b9da73f667173fa in term 0.
I20260925 22:00:57.869541 16961 raft_consensus.cc:2454] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a083e388c8dd42708b9da73f667173fa in term 0.
I20260925 22:00:57.869760 16760 leader_election.cc:302] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2254d97d517b49adb1d96b2817bd98bc, 36cedb52a1e74631bffaff0958d4dfa3, a083e388c8dd42708b9da73f667173fa; no voters:
I20260925 22:00:57.869939 17104 raft_consensus.cc:2781] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:57.869992 17104 raft_consensus.cc:482] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:57.870020 17104 raft_consensus.cc:3037] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:57.870934 17104 raft_consensus.cc:504] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.871095 17104 leader_election.cc:288] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [CANDIDATE]: Term 1 election: Requested vote from peers 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809), 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545)
I20260925 22:00:57.871285 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edb6064d04842e5a975c7cdecb0d292" candidate_uuid: "a083e388c8dd42708b9da73f667173fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2254d97d517b49adb1d96b2817bd98bc"
I20260925 22:00:57.871524 16688 raft_consensus.cc:3037] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:57.872313 16688 raft_consensus.cc:2454] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a083e388c8dd42708b9da73f667173fa in term 1.
I20260925 22:00:57.872587 16961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edb6064d04842e5a975c7cdecb0d292" candidate_uuid: "a083e388c8dd42708b9da73f667173fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cedb52a1e74631bffaff0958d4dfa3"
I20260925 22:00:57.872664 16761 leader_election.cc:302] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2254d97d517b49adb1d96b2817bd98bc, a083e388c8dd42708b9da73f667173fa; no voters:
I20260925 22:00:57.872715 16961 raft_consensus.cc:3037] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:57.872805 17104 raft_consensus.cc:2781] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:57.873054 17104 raft_consensus.cc:686] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 1 LEADER]: Becoming Leader. State: Replica: a083e388c8dd42708b9da73f667173fa, State: Running, Role: LEADER
I20260925 22:00:57.873147 17104 consensus_queue.cc:237] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } }
I20260925 22:00:57.873581 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.873424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:39491)
0925 22:00:57.873493 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.873543 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 22:00:57.873701 16961 raft_consensus.cc:2454] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a083e388c8dd42708b9da73f667173fa in term 1.
I20260925 22:00:57.874202 16555 catalog_manager.cc:5495] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa reported cstate change: term changed from 0 to 1, leader changed from <none> to a083e388c8dd42708b9da73f667173fa (127.9.112.66). New cstate: current_term: 1 leader_uuid: "a083e388c8dd42708b9da73f667173fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a083e388c8dd42708b9da73f667173fa" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 44873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 } health_report { overall_health: UNKNOWN } } }
I20260925 22:00:57.902601 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.900201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.9.112.66:44873)
0925 22:00:57.900271 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:57.900274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:57.900282 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:57.900327 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:57.900328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:57.900390 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:57.900458 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:57.900578 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.901103 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.901877 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:57.902067 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:57.902122 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:57.902282 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:57.902392 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:57.902455 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:57.902494 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 22:00:57.902514 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 22:00:57.904083 16688 raft_consensus.cc:1261] T 8edb6064d04842e5a975c7cdecb0d292 P 2254d97d517b49adb1d96b2817bd98bc [term 1 FOLLOWER]: Refusing update from remote peer a083e388c8dd42708b9da73f667173fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:57.904127 16961 raft_consensus.cc:1261] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Refusing update from remote peer a083e388c8dd42708b9da73f667173fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:57.904480 17113 consensus_queue.cc:1035] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "36cedb52a1e74631bffaff0958d4dfa3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:57.904635 17104 consensus_queue.cc:1035] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2254d97d517b49adb1d96b2817bd98bc" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:57.905495 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.905365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:44549)
0925 22:00:57.905424 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.905459 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:00:57.905555 16734 heartbeater.cc:503] Master 127.9.112.124:34087 was elected leader, sending a full tablet report...
I20260925 22:00:57.905860 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.905751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:48523)
0925 22:00:57.905797 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.905820 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 22:00:57.907246 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.907099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.67:57189)
0925 22:00:57.907160 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.907202 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 22:00:57.907603 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.907077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:55929)
0925 22:00:57.907535 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.907569 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439}
I20260925 22:00:57.908020 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.907908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.66:48683)
0925 22:00:57.907953 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.907980 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 22:00:57.908867 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.908781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43433 (local address 127.9.112.65:42509)
0925 22:00:57.908815 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.908837 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 22:00:57.909672 16555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:57.909781 16555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:57.911229 16555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8eb490f50c1409db6576b612a354b5c: Sending DeleteTablet for 3 replicas of tablet 8edb6064d04842e5a975c7cdecb0d292
I20260925 22:00:57.911639 16941 tablet_service.cc:1514] Processing DeleteTablet for tablet 8edb6064d04842e5a975c7cdecb0d292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:57 UTC) from {username='oryx', principal='oryx/127.9.112.124@KRBTEST.COM'} at 127.0.0.1:35568
I20260925 22:00:57.911813 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 16516
I20260925 22:00:57.911908 16668 tablet_service.cc:1514] Processing DeleteTablet for tablet 8edb6064d04842e5a975c7cdecb0d292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:57 UTC) from {username='oryx', principal='oryx/127.9.112.124@KRBTEST.COM'} at 127.0.0.1:48018
I20260925 22:00:57.912443 17125 tablet_replica.cc:330] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: stopping tablet replica
I20260925 22:00:57.912503 17125 raft_consensus.cc:2229] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:57.912554 17125 raft_consensus.cc:2258] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:57.912576 16805 tablet_service.cc:1514] Processing DeleteTablet for tablet 8edb6064d04842e5a975c7cdecb0d292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:57 UTC) from {username='oryx', principal='oryx/127.9.112.124@KRBTEST.COM'} at 127.0.0.1:56078
I20260925 22:00:57.912823 17126 tablet_replica.cc:330] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: stopping tablet replica
I20260925 22:00:57.912878 17126 raft_consensus.cc:2229] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:57.912971 17126 raft_consensus.cc:2258] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:57.913311 17126 ts_tablet_manager.cc:1905] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:57.914299 17125 ts_tablet_manager.cc:1905] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:57.915380 17126 ts_tablet_manager.cc:1918] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:57.915450 17126 log.cc:1196] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/8edb6064d04842e5a975c7cdecb0d292
I20260925 22:00:57.915685 17126 ts_tablet_manager.cc:1939] T 8edb6064d04842e5a975c7cdecb0d292 P a083e388c8dd42708b9da73f667173fa: Deleting consensus metadata
I20260925 22:00:57.916213 17125 ts_tablet_manager.cc:1918] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:57.916500 16542 catalog_manager.cc:4841] TS a083e388c8dd42708b9da73f667173fa (127.9.112.66:44873): tablet 8edb6064d04842e5a975c7cdecb0d292 (table test-table [id=de3dec0317f641b3a00e5159282c475e]) successfully deleted
I20260925 22:00:57.916558 17125 log.cc:1196] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/8edb6064d04842e5a975c7cdecb0d292
I20260925 22:00:57.916841 17125 ts_tablet_manager.cc:1939] T 8edb6064d04842e5a975c7cdecb0d292 P 36cedb52a1e74631bffaff0958d4dfa3: Deleting consensus metadata
I20260925 22:00:57.917330 16541 catalog_manager.cc:4841] TS 36cedb52a1e74631bffaff0958d4dfa3 (127.9.112.67:43545): tablet 8edb6064d04842e5a975c7cdecb0d292 (table test-table [id=de3dec0317f641b3a00e5159282c475e]) successfully deleted
I20260925 22:00:57.918686 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:57.918552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42809 (local address 127.0.0.1:48024)
0925 22:00:57.918612 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:57.918642 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:00:57.918694 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 16742
W20260925 22:00:57.919440 16542 catalog_manager.cc:4584] TS 2254d97d517b49adb1d96b2817bd98bc (127.9.112.65:42809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8edb6064d04842e5a975c7cdecb0d292: Network error: Client connection negotiation failed: client connection to 127.9.112.65:42809: connect: Connection refused (error 111)
I20260925 22:00:57.924366 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 16878
I20260925 22:00:57.930125 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 16235
I20260925 22:00:57.935370 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 16423
2026-09-25T22:00:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4883 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:00:57 dist-test-slave-794d krb5kdc[17129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:00:57 dist-test-slave-794d krb5kdc[17129](info): set up 2 sockets
Sep 25 22:00:57 dist-test-slave-794d krb5kdc[17129](info): commencing operation
krb5kdc: starting...
W20260925 22:00:58.904673 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:00:58 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:00:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab.
Entry for principal oryx/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab.
I20260925 22:00:59.029111 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:33253
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab
--principal=oryx/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:33253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:59.110947 17142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:59.111173 17142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:59.111197 17142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:59.112495 17142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:00:59.112555 17142 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 22:00:59.112567 17142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:59.112579 17142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:59.112591 17142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:00:59.112601 17142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:00:59.114322 17142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:33253
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.126.keytab
--principal=oryx/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:33253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:59.114589 17142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:59.114885 17142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:59.119729 17147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.119724 17150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.119735 17148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:59.119920 17142 server_base.cc:1034] running on GCE node
I20260925 22:00:59.120205 17142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:59.120535 17142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:59.121701 17142 hybrid_clock.cc:648] HybridClock initialized: now 1790373659121684 us; error 29 us; skew 500 ppm
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:59.124615 17142 init.cc:377] Logged in from keytab as oryx/127.9.112.126@KRBTEST.COM (short username oryx)
I20260925 22:00:59.125159 17142 webserver.cc:460] Webserver started at http://127.9.112.126:46237/ using document root <none> and password file <none>
I20260925 22:00:59.125389 17142 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:59.125442 17142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:59.125571 17142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:59.126628 17142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "dfc93849005b4ea5950e5418175af6bf"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "16465c0725f1e2ac6d46a5612ab74049"
server_key_iv: "a93d93f4f23252c6f7947b6188befa70"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.126964 17142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "dfc93849005b4ea5950e5418175af6bf"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "16465c0725f1e2ac6d46a5612ab74049"
server_key_iv: "a93d93f4f23252c6f7947b6188befa70"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.128283 17142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:59.129077 17157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.129226 17142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:59.129320 17142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "dfc93849005b4ea5950e5418175af6bf"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "16465c0725f1e2ac6d46a5612ab74049"
server_key_iv: "a93d93f4f23252c6f7947b6188befa70"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.129384 17142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:59.151707 17142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:59.154253 17142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:59.154412 17142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:59.159595 17142 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:33253
I20260925 22:00:59.159771 17219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:33253 every 8 connection(s)
I20260925 22:00:59.160096 17142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:00:59.160967 17220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.163436 17220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Bootstrap starting.
I20260925 22:00:59.164095 17220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.164433 17220 log.cc:824] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:59.164705 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 17142
I20260925 22:00:59.164793 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:00:59.164948 9665 external_mini_cluster.cc:1428] Setting key 3c6c762d0fdbc886476c8f4b009d6a63
I20260925 22:00:59.165292 17220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: No bootstrap required, opened a new log
I20260925 22:00:59.167996 17220 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc93849005b4ea5950e5418175af6bf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33253 } }
I20260925 22:00:59.168213 17220 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.168249 17220 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc93849005b4ea5950e5418175af6bf, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.168418 17220 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc93849005b4ea5950e5418175af6bf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33253 } }
I20260925 22:00:59.168501 17220 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:00:59.168530 17220 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:00:59.168567 17220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:59.169279 17220 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc93849005b4ea5950e5418175af6bf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33253 } }
I20260925 22:00:59.169376 17220 leader_election.cc:302] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfc93849005b4ea5950e5418175af6bf; no voters:
I20260925 22:00:59.169550 17220 leader_election.cc:288] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:00:59.169581 17225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:59.169751 17225 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [term 1 LEADER]: Becoming Leader. State: Replica: dfc93849005b4ea5950e5418175af6bf, State: Running, Role: LEADER
I20260925 22:00:59.169831 17220 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:00:59.169859 17225 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc93849005b4ea5950e5418175af6bf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33253 } }
I20260925 22:00:59.170210 17225 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfc93849005b4ea5950e5418175af6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc93849005b4ea5950e5418175af6bf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33253 } } }
I20260925 22:00:59.170388 17225 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:59.170575 17225 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfc93849005b4ea5950e5418175af6bf. Latest consensus state: current_term: 1 leader_uuid: "dfc93849005b4ea5950e5418175af6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc93849005b4ea5950e5418175af6bf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33253 } } }
I20260925 22:00:59.170684 17225 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf [sys.catalog]: This master's current role is: LEADER
I20260925 22:00:59.170837 17234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:00:59.171546 17234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:59.173520 17234 catalog_manager.cc:1347] Generated new cluster ID: 9bc39b69c68547a0ae92ad9a9cfea495
I20260925 22:00:59.173573 17234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:00:59.177625 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.165530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.9.112.126:33253)
0925 22:00:59.165722 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.165732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.165770 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:00:59.166597 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.166608 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.166696 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.166866 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.167486 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.168833 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.169769 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.170443 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.170567 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.173395 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.173400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.173400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.173415 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.175510 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.175968 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.175971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.175971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.176081 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.176288 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.176289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.176289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.176641 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.176793 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.177255 (+ 462us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.177410 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260925 22:00:59.191169 17234 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:00:59.191987 17234 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:00:59.202934 17234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Generated new TSK 0
I20260925 22:00:59.203167 17234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab.
Entry for principal oryx/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab.
I20260925 22:00:59.233448 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33253
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:59.319609 17246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:59.319862 17246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:59.319890 17246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:59.321381 17246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:59.321445 17246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:59.321470 17246 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:59.321487 17246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:00:59.323258 17246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:33253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:59.323523 17246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:59.323788 17246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:59.329627 17252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.329653 17251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.329640 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:59.329959 17246 server_base.cc:1034] running on GCE node
I20260925 22:00:59.330178 17246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:59.330474 17246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:59.331595 17246 hybrid_clock.cc:648] HybridClock initialized: now 1790373659331590 us; error 39 us; skew 500 ppm
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:59.334317 17246 init.cc:377] Logged in from keytab as oryx/127.9.112.65@KRBTEST.COM (short username oryx)
I20260925 22:00:59.334782 17246 webserver.cc:460] Webserver started at http://127.9.112.65:36151/ using document root <none> and password file <none>
I20260925 22:00:59.335050 17246 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:59.335098 17246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:59.335207 17246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:59.336325 17246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "8dd6a102a4714666810a286816c08484"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "56be8d40760aa6261157fb301236dd08"
server_key_iv: "bcb465be4438302259673ea636c271fd"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.336723 17246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "8dd6a102a4714666810a286816c08484"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "56be8d40760aa6261157fb301236dd08"
server_key_iv: "bcb465be4438302259673ea636c271fd"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.338075 17246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:59.338876 17261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.339030 17246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:00:59.339123 17246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "8dd6a102a4714666810a286816c08484"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "56be8d40760aa6261157fb301236dd08"
server_key_iv: "bcb465be4438302259673ea636c271fd"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.339187 17246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:59.360330 17246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:59.362769 17246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:59.362977 17246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:59.363509 17246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:59.363554 17246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.363585 17246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:59.363602 17246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:59.370877 17246 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:37835
I20260925 22:00:59.370983 17376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:37835 every 8 connection(s)
I20260925 22:00:59.371280 17246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:00:59.373677 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.364004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40885 (local address 127.9.112.126:33253)
0925 22:00:59.364046 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.364048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.364283 (+ 235us) server_negotiation.cc:407] Connection header received
0925 22:00:59.364876 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.364877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.364890 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.364950 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.365850 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.366378 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.367952 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.368175 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.368238 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.370849 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.370851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.370852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.370861 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.372364 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.372751 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.372752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.372752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.372820 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.373040 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.373041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.373041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.373162 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.373216 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.373493 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.373585 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:00:59.373850 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.364003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.9.112.65:40885)
0925 22:00:59.364164 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.364200 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.364734 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.365063 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.365068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.365450 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:59.365751 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.365756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.366449 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.366450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.367858 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.367862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.368286 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.368287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.368452 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.369044 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:59.369054 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:59.370730 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:59.372446 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.372454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.372456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.372682 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.372876 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.372876 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.372877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.372986 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.373214 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:59.373220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:59.373374 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.373466 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.373631 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260925 22:00:59.374291 17377 heartbeater.cc:347] Connected to a master server at 127.9.112.126:33253
I20260925 22:00:59.374382 17377 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:59.374567 17377 heartbeater.cc:511] Master 127.9.112.126:33253 requested a full tablet report, sending...
I20260925 22:00:59.375088 17173 ts_manager.cc:194] Registered new tserver with Master: 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835)
I20260925 22:00:59.375931 17173 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:40885
I20260925 22:00:59.379357 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 17246
I20260925 22:00:59.379554 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:00:59.379767 9665 external_mini_cluster.cc:1428] Setting key 7c94a76a5c208c0c3b7dd11a381cf722
I20260925 22:00:59.381073 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.375278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:35271 (local address 127.9.112.126:33253)
0925 22:00:59.375332 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.375334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.375400 (+ 66us) server_negotiation.cc:407] Connection header received
0925 22:00:59.375448 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.375449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.375457 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.375521 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.375843 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.376569 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.377392 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.377580 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.377641 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.378579 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.378580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.378581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.378587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.380015 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.380376 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.380377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.380378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.380433 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.380602 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.380603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.380603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.380723 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.380792 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.380943 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.381009 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:00:59.381127 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.375226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.9.112.65:35271)
0925 22:00:59.375351 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.375360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.375407 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.375527 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.375527 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.375673 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:59.375731 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.375732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.376660 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.376661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.377290 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.377293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.377651 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.377652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.377703 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.378190 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:59.378194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:59.378492 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:59.380091 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.380092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.380093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.380271 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.380486 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.380487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.380488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.380543 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.380796 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:59.380797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:59.380853 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.380921 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.381029 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for oryx/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab.
Entry for principal oryx/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab.
I20260925 22:00:59.407733 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33253
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:59.493141 17386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:59.493357 17386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:59.493383 17386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:59.494591 17386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:59.494639 17386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:59.494660 17386 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:59.494678 17386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:00:59.496225 17386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:33253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:59.496490 17386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:59.496712 17386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:59.502674 17394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.502648 17392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.502745 17391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:59.502821 17386 server_base.cc:1034] running on GCE node
I20260925 22:00:59.503328 17386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:59.503607 17386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:59.504757 17386 hybrid_clock.cc:648] HybridClock initialized: now 1790373659504733 us; error 44 us; skew 500 ppm
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:59.507299 17386 init.cc:377] Logged in from keytab as oryx/127.9.112.66@KRBTEST.COM (short username oryx)
I20260925 22:00:59.507743 17386 webserver.cc:460] Webserver started at http://127.9.112.66:45085/ using document root <none> and password file <none>
I20260925 22:00:59.507965 17386 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:59.508013 17386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:59.508147 17386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:59.509210 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "477f8b24c46f4e33996c53ca48f402f8"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "357b8298a8d6f8f48dc1187c1d9c5889"
server_key_iv: "80b4ad8c4220ce6cf4537a5e8c895691"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.509541 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "477f8b24c46f4e33996c53ca48f402f8"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "357b8298a8d6f8f48dc1187c1d9c5889"
server_key_iv: "80b4ad8c4220ce6cf4537a5e8c895691"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.510841 17386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:00:59.511538 17401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.511685 17386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:00:59.511767 17386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "477f8b24c46f4e33996c53ca48f402f8"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "357b8298a8d6f8f48dc1187c1d9c5889"
server_key_iv: "80b4ad8c4220ce6cf4537a5e8c895691"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.511833 17386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:59.529057 17386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:59.531380 17386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:59.531518 17386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:59.532133 17386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:59.532171 17386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.532191 17386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:59.532202 17386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.538640 17386 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:42991
I20260925 22:00:59.538712 17516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:42991 every 8 connection(s)
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:59.539018 17386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:00:59.541883 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.532586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:54457 (local address 127.9.112.126:33253)
0925 22:00:59.532644 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.532647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.532814 (+ 167us) server_negotiation.cc:407] Connection header received
0925 22:00:59.533410 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.533412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.533424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.533502 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.534559 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.535370 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.536916 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.537129 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.537189 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.539585 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.539587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.539587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.539595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.540838 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.541184 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.541184 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.541184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.541229 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.541410 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.541411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.541411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.541509 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.541553 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.541760 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.541821 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:00:59.542104 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.532557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.9.112.66:54457)
0925 22:00:59.532698 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.532741 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.533301 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.533573 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.533577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.533931 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:59.534452 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.534463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.535463 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.535464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.536807 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.536811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.537204 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.537205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.537344 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.537939 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:59.537948 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:59.539497 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:59.540907 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.540912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.540914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.541127 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.541272 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.541272 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.541273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.541361 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.541548 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:59.541551 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:59.541666 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.541739 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.541883 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260925 22:00:59.542505 17517 heartbeater.cc:347] Connected to a master server at 127.9.112.126:33253
I20260925 22:00:59.542588 17517 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:59.542762 17517 heartbeater.cc:511] Master 127.9.112.126:33253 requested a full tablet report, sending...
I20260925 22:00:59.543045 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 17386
I20260925 22:00:59.543123 17174 ts_manager.cc:194] Registered new tserver with Master: 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991)
I20260925 22:00:59.543164 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:00:59.543341 9665 external_mini_cluster.cc:1428] Setting key 1f51a8b282fcd2dea7eb325637b672a3
I20260925 22:00:59.543597 17174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:54457
I20260925 22:00:59.550549 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.543536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.9.112.66:54009)
0925 22:00:59.544470 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.544480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.544537 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.544656 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.544657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.544824 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:59.544880 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.544882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.545624 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.545625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.546382 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.546384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.546837 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.546838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.546906 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.547343 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:59.547348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:59.547640 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:59.549422 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.549423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.549424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.549590 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.549777 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.549778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.549778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.549832 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.550254 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:59.550255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:59.550305 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.550371 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.550473 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":80,"threads_started":1}
I20260925 22:00:59.550591 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.543625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:54009 (local address 127.9.112.126:33253)
0925 22:00:59.543670 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.543672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.544547 (+ 875us) server_negotiation.cc:407] Connection header received
0925 22:00:59.544572 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.544573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.544585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.544648 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.544940 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.545539 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.546472 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.546744 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.546814 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.547780 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.547781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.547782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.547790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.549322 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.549658 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.549659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.549659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.549719 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.549887 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.549888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.549888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.550094 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.550167 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.550415 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.550498 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for oryx/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab.
Entry for principal oryx/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab.
I20260925 22:00:59.571967 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33253
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:00:59.655143 17526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:00:59.655369 17526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:00:59.655393 17526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:00:59.656587 17526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:00:59.656626 17526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:00:59.656653 17526 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 22:00:59.656672 17526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:00:59.658179 17526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.112.126:33253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:00:59.658411 17526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:00:59.658629 17526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:00:59.663698 17532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.663678 17531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:00:59.663740 17534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:00:59.664021 17526 server_base.cc:1034] running on GCE node
I20260925 22:00:59.664227 17526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:00:59.664505 17526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:00:59.665710 17526 hybrid_clock.cc:648] HybridClock initialized: now 1790373659665696 us; error 27 us; skew 500 ppm
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:00:59.668812 17526 init.cc:377] Logged in from keytab as oryx/127.9.112.67@KRBTEST.COM (short username oryx)
I20260925 22:00:59.669389 17526 webserver.cc:460] Webserver started at http://127.9.112.67:45305/ using document root <none> and password file <none>
I20260925 22:00:59.669617 17526 fs_manager.cc:359] Metadata directory not provided
I20260925 22:00:59.669664 17526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:00:59.669889 17526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:00:59.671079 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "83754f2353cb49d6b6b7f92c8950bf50"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "32b289e0ecfdf6a330abc084cdfd677b"
server_key_iv: "891a2620ab0474bfc562543edfa14d84"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.671443 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "83754f2353cb49d6b6b7f92c8950bf50"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "32b289e0ecfdf6a330abc084cdfd677b"
server_key_iv: "891a2620ab0474bfc562543edfa14d84"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.672840 17526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:00:59.673647 17541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.673887 17526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:59.673977 17526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "83754f2353cb49d6b6b7f92c8950bf50"
format_stamp: "Formatted at 2026-09-25 22:00:59 on dist-test-slave-794d"
server_key: "32b289e0ecfdf6a330abc084cdfd677b"
server_key_iv: "891a2620ab0474bfc562543edfa14d84"
server_key_version: "encryptionkey@0"
I20260925 22:00:59.674039 17526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:00:59.693552 17526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:00:59.695724 17526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:00:59.695973 17526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:00:59.696606 17526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:00:59.696667 17526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.696699 17526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:00:59.696715 17526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:59.703743 17526 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:46301
I20260925 22:00:59.704092 17526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:00:59.704109 17656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:46301 every 8 connection(s)
I20260925 22:00:59.706315 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.697079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:52405 (local address 127.9.112.126:33253)
0925 22:00:59.697151 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.697153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.697338 (+ 185us) server_negotiation.cc:407] Connection header received
0925 22:00:59.697799 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.697800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.697813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.697889 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.698683 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.699449 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.700620 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.700907 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.700981 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.703238 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.703239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.703240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.703249 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.704800 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.705240 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.705242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.705242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.705298 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.705581 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.705582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.705582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.705710 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.705838 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.706176 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.706246 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:00:59.706423 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.697061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.9.112.67:52405)
0925 22:00:59.697208 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.697248 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.697712 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.697949 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.697953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.698294 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:59.698589 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.698596 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.699581 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.699582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.700535 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.700539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.701008 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.701010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.701150 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.701654 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:59.701662 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:59.703159 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:59.704905 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.704911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.704913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.705142 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.705381 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.705382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.705382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.705495 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.705863 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:59.705869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:59.705997 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.706101 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.706241 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260925 22:00:59.706871 17657 heartbeater.cc:347] Connected to a master server at 127.9.112.126:33253
I20260925 22:00:59.706954 17657 heartbeater.cc:464] Registering TS with master...
I20260925 22:00:59.707126 17657 heartbeater.cc:511] Master 127.9.112.126:33253 requested a full tablet report, sending...
I20260925 22:00:59.707204 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 17526
I20260925 22:00:59.707305 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:00:59.707448 17174 ts_manager.cc:194] Registered new tserver with Master: 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:00:59.707510 9665 external_mini_cluster.cc:1428] Setting key 1898a3cac6d7dc891a81eaaee7d74d51
I20260925 22:00:59.707974 17174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:52405
I20260925 22:00:59.708266 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:00:59.715443 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.707942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:46701 (local address 127.9.112.126:33253)
0925 22:00:59.707992 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.707994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.708119 (+ 125us) server_negotiation.cc:407] Connection header received
0925 22:00:59.708196 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.708197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.708207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.708247 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.710087 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.710613 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.711554 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.711770 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.711834 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.712880 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.712882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.712882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.712890 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.714378 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.714757 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.714759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.714759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.714826 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.715011 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.715011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.715012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.715113 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.715196 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.715313 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.715376 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:00:59.715515 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.707888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.9.112.67:46701)
0925 22:00:59.708045 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.708055 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.708144 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.709783 (+ 1639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.709784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.709945 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:00:59.710000 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.710002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.710715 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.710716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.711450 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.711453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.711850 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.711851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.711916 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.712396 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:00:59.712401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:00:59.712792 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:00:59.714438 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.714439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.714439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.714660 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.714891 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:00:59.714892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:00:59.714892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:00:59.714960 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:00:59.715169 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:00:59.715169 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:00:59.715220 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.715292 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.715394 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":58,"threads_started":1}
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:00:59.730160 17224 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:00:59.724875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.9.112.126:33253)
0925 22:00:59.724924 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.724925 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.724933 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:59.724958 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.724959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.724967 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.725023 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.725637 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.726161 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.726875 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.727050 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.727093 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.730070 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52920: BlockingRecv error: recv got EOF from 127.0.0.1:52920 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 25 22:00:59 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:00:59.737321 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.730980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52928 (local address 127.9.112.126:33253)
0925 22:00:59.731042 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.731044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.731053 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:59.731116 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.731117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.731126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.731171 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:00:59.731627 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.732111 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.732792 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.732980 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.733032 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.735060 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:00:59.735061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:00:59.735062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:00:59.735069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:00:59.736130 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.736484 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.736484 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.736485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.736534 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:00:59.736838 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:00:59.736840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:00:59.736840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:00:59.736968 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:00:59.737061 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.737182 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.737254 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:00:59.739106 17173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:59.740664 17173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:59.747433 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.743384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36742 (local address 127.9.112.67:46301)
0925 22:00:59.743554 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.743558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.743992 (+ 434us) server_negotiation.cc:407] Connection header received
0925 22:00:59.744036 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.744038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.744053 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.744146 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:59.744243 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.745242 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.746175 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.746938 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.747030 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.747137 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.747275 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.747319 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260925 22:00:59.747435 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.743013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37835 (local address 127.0.0.1:32994)
0925 22:00:59.743178 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.743215 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.743281 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.743572 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.743576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.743578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:59.743741 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.743744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.745060 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.745060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.746194 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.746197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.747164 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.747165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.747235 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.747240 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.747297 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.747343 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260925 22:00:59.747447 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.743034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42991 (local address 127.0.0.1:49090)
0925 22:00:59.743357 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.743367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.743417 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.743776 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.743777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.743780 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:59.743851 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.743853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.744906 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.744907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.746419 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.746420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.747182 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.747183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.747235 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.747239 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.747281 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.747310 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260925 22:00:59.747447 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.743242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:46301 (local address 127.0.0.1:36742)
0925 22:00:59.743872 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.743883 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.743932 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.744142 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.744143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.744145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:59.744181 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.744182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.745322 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.745323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.746077 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.746079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.747074 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.747075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.747158 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.747174 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.747247 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.747305 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":65,"threads_started":1}
I20260925 22:00:59.747864 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.743106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.9.112.65:37835)
0925 22:00:59.743391 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.743396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.743405 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:59.743440 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.743442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.743453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.743525 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:59.743835 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.744968 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.746276 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.746985 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.747050 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.747121 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.747660 (+ 539us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.747703 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
I20260925 22:00:59.748168 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.743192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.9.112.66:42991)
0925 22:00:59.743595 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.743600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.743610 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:00:59.743649 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.743651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.743668 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.743778 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:59.743933 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.744818 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.746486 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.747003 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.747073 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.747151 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.747483 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.748069 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":83,"threads_started":1}
I20260925 22:00:59.748517 17311 tablet_service.cc:1467] Processing CreateTablet for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd (DEFAULT_TABLE table=test-table [id=c4eeb1ffe67b43dc95e249c5ef9a40cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:59.748558 17591 tablet_service.cc:1467] Processing CreateTablet for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd (DEFAULT_TABLE table=test-table [id=c4eeb1ffe67b43dc95e249c5ef9a40cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:59.748939 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.748939 17311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.748893 17451 tablet_service.cc:1467] Processing CreateTablet for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd (DEFAULT_TABLE table=test-table [id=c4eeb1ffe67b43dc95e249c5ef9a40cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:59.749207 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.751881 17685 tablet_bootstrap.cc:492] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Bootstrap starting.
I20260925 22:00:59.752601 17685 tablet_bootstrap.cc:654] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.752885 17685 log.cc:824] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:59.753090 17686 tablet_bootstrap.cc:492] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Bootstrap starting.
I20260925 22:00:59.753614 17688 tablet_bootstrap.cc:492] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Bootstrap starting.
I20260925 22:00:59.753713 17686 tablet_bootstrap.cc:654] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.753741 17685 tablet_bootstrap.cc:492] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: No bootstrap required, opened a new log
I20260925 22:00:59.753810 17685 ts_tablet_manager.cc:1397] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:59.754017 17686 log.cc:824] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:59.754223 17688 tablet_bootstrap.cc:654] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.754456 17688 log.cc:824] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Log is configured to *not* fsync() on all Append() calls
I20260925 22:00:59.754805 17686 tablet_bootstrap.cc:492] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: No bootstrap required, opened a new log
I20260925 22:00:59.754866 17686 ts_tablet_manager.cc:1397] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:59.755223 17688 tablet_bootstrap.cc:492] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: No bootstrap required, opened a new log
I20260925 22:00:59.755334 17688 ts_tablet_manager.cc:1397] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:00:59.755650 17685 raft_consensus.cc:348] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.755801 17685 raft_consensus.cc:374] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.755836 17685 raft_consensus.cc:729] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83754f2353cb49d6b6b7f92c8950bf50, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.755959 17685 consensus_queue.cc:260] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.756235 17685 ts_tablet_manager.cc:1428] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:00:59.756426 17657 heartbeater.cc:503] Master 127.9.112.126:33253 was elected leader, sending a full tablet report...
I20260925 22:00:59.757064 17686 raft_consensus.cc:348] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.757236 17686 raft_consensus.cc:374] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.757284 17686 raft_consensus.cc:729] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.757441 17686 consensus_queue.cc:260] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.757440 17688 raft_consensus.cc:348] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.757575 17688 raft_consensus.cc:374] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.757617 17688 raft_consensus.cc:729] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6a102a4714666810a286816c08484, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.757722 17686 ts_tablet_manager.cc:1428] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:00:59.757727 17688 consensus_queue.cc:260] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.757835 17517 heartbeater.cc:503] Master 127.9.112.126:33253 was elected leader, sending a full tablet report...
I20260925 22:00:59.757990 17688 ts_tablet_manager.cc:1428] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:00:59.758023 17377 heartbeater.cc:503] Master 127.9.112.126:33253 was elected leader, sending a full tablet report...
W20260925 22:00:59.789599 17518 tablet.cc:2369] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:00:59.801017 17691 raft_consensus.cc:482] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:00:59.801142 17691 raft_consensus.cc:504] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.801525 17691 leader_election.cc:288] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991), 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835)
I20260925 22:00:59.805573 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.801893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:34179 (local address 127.9.112.66:42991)
0925 22:00:59.801937 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.801939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.801947 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:00:59.801973 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.801974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.801982 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.802036 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:59.802293 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.803045 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.804594 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.805142 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.805224 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.805234 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.805457 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.805497 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:00:59.805573 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.801946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:58705 (local address 127.9.112.65:37835)
0925 22:00:59.801995 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.801997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.802006 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:00:59.802031 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.802032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.802040 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.802089 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:00:59.802273 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.803052 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.804587 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.805143 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.805224 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.805234 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.805451 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.805497 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:00:59.805586 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.801743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42991 (local address 127.9.112.67:34179)
0925 22:00:59.801804 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.801817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.801880 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.802061 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.802062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.802063 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:59.802140 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.802142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.803185 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.803185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.804497 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.804501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.805250 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.805251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.805369 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.805371 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.805424 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.805470 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":17}
I20260925 22:00:59.805872 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.801743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37835 (local address 127.9.112.67:58705)
0925 22:00:59.801923 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:00:59.801934 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:00:59.801987 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:00:59.802099 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:00:59.802100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:00:59.802101 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:00:59.802157 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.802158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.803166 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.803167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.804498 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:00:59.804501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.805250 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.805251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:00:59.805352 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.805358 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:00:59.805432 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:00:59.805470 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":66,"thread_start_us":100,"threads_started":1}
I20260925 22:00:59.805922 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b701fad8e74b5f9ea7cd1c8c0e37dd" candidate_uuid: "83754f2353cb49d6b6b7f92c8950bf50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477f8b24c46f4e33996c53ca48f402f8" is_pre_election: true
I20260925 22:00:59.806080 17471 raft_consensus.cc:2454] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83754f2353cb49d6b6b7f92c8950bf50 in term 0.
I20260925 22:00:59.806180 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b701fad8e74b5f9ea7cd1c8c0e37dd" candidate_uuid: "83754f2353cb49d6b6b7f92c8950bf50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484" is_pre_election: true
I20260925 22:00:59.806321 17331 raft_consensus.cc:2454] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83754f2353cb49d6b6b7f92c8950bf50 in term 0.
I20260925 22:00:59.806376 17543 leader_election.cc:302] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 83754f2353cb49d6b6b7f92c8950bf50; no voters:
I20260925 22:00:59.806509 17691 raft_consensus.cc:2781] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:00:59.806573 17691 raft_consensus.cc:482] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:00:59.806591 17691 raft_consensus.cc:3037] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:59.807600 17691 raft_consensus.cc:504] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.807745 17691 leader_election.cc:288] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [CANDIDATE]: Term 1 election: Requested vote from peers 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991), 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835)
I20260925 22:00:59.807948 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b701fad8e74b5f9ea7cd1c8c0e37dd" candidate_uuid: "83754f2353cb49d6b6b7f92c8950bf50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484"
I20260925 22:00:59.807962 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b701fad8e74b5f9ea7cd1c8c0e37dd" candidate_uuid: "83754f2353cb49d6b6b7f92c8950bf50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477f8b24c46f4e33996c53ca48f402f8"
I20260925 22:00:59.808022 17331 raft_consensus.cc:3037] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:59.808053 17471 raft_consensus.cc:3037] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:00:59.808750 17331 raft_consensus.cc:2454] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83754f2353cb49d6b6b7f92c8950bf50 in term 1.
I20260925 22:00:59.808880 17471 raft_consensus.cc:2454] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83754f2353cb49d6b6b7f92c8950bf50 in term 1.
I20260925 22:00:59.808959 17545 leader_election.cc:302] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83754f2353cb49d6b6b7f92c8950bf50, 8dd6a102a4714666810a286816c08484; no voters:
I20260925 22:00:59.809047 17691 raft_consensus.cc:2781] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:00:59.809198 17691 raft_consensus.cc:686] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 LEADER]: Becoming Leader. State: Replica: 83754f2353cb49d6b6b7f92c8950bf50, State: Running, Role: LEADER
I20260925 22:00:59.809310 17691 consensus_queue.cc:237] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.810194 17173 catalog_manager.cc:5495] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67). New cstate: current_term: 1 leader_uuid: "83754f2353cb49d6b6b7f92c8950bf50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } health_report { overall_health: HEALTHY } } }
I20260925 22:00:59.826053 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:00:59.823798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.9.112.67:46301)
0925 22:00:59.823859 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:00:59.823860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:00:59.823867 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:00:59.823899 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:00:59.823900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:00:59.823917 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:00:59.823976 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:00:59.824104 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.824621 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.825354 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:00:59.825537 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:00:59.825590 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:00:59.825736 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:00:59.825846 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:00:59.825914 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:00:59.825953 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 22:00:59.825974 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:00:59.827867 17331 raft_consensus.cc:1261] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Refusing update from remote peer 83754f2353cb49d6b6b7f92c8950bf50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:59.827886 17471 raft_consensus.cc:1261] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Refusing update from remote peer 83754f2353cb49d6b6b7f92c8950bf50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:00:59.828209 17695 consensus_queue.cc:1035] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:59.828406 17691 consensus_queue.cc:1035] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:00:59.830938 17699 mvcc.cc:205] Tried to move back new op lower bound from 7333370510652329984 to 7333370510579761152. Current Snapshot: MvccSnapshot[applied={T|T < 7333370510652329984}]
I20260925 22:00:59.831447 17700 mvcc.cc:205] Tried to move back new op lower bound from 7333370510652329984 to 7333370510579761152. Current Snapshot: MvccSnapshot[applied={T|T < 7333370510652329984}]
I20260925 22:00:59.833312 17174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:59.833415 17174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:00:59.834224 17174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Sending DeleteTablet for 3 replicas of tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd
I20260925 22:00:59.834559 17311 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:59 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:32994
I20260925 22:00:59.834626 17591 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:59 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:36742
I20260925 22:00:59.834766 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:00:59 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49090
I20260925 22:00:59.834860 17707 tablet_replica.cc:330] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: stopping tablet replica
I20260925 22:00:59.834960 17707 raft_consensus.cc:2229] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:59.835094 17174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:00:59.835249 17174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:00:59.835311 17707 raft_consensus.cc:2258] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:59.835559 17709 tablet_replica.cc:330] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: stopping tablet replica
I20260925 22:00:59.835883 17707 ts_tablet_manager.cc:1905] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:59.836571 17311 tablet_service.cc:1467] Processing CreateTablet for tablet 2899a3813b0f4ceb8b3a14a933b2c326 (DEFAULT_TABLE table=test-table [id=b7ecdad01bb94e13b74934902ff827d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:59.835553 17708 tablet_replica.cc:330] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: stopping tablet replica
I20260925 22:00:59.836683 17311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2899a3813b0f4ceb8b3a14a933b2c326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.836692 17708 raft_consensus.cc:2229] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:00:59.836864 17708 raft_consensus.cc:2258] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:59.837275 17708 ts_tablet_manager.cc:1905] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:59.837437 17709 raft_consensus.cc:2229] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:00:59.837528 17591 tablet_service.cc:1467] Processing CreateTablet for tablet 2899a3813b0f4ceb8b3a14a933b2c326 (DEFAULT_TABLE table=test-table [id=b7ecdad01bb94e13b74934902ff827d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:59.837568 17709 raft_consensus.cc:2258] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:00:59.837653 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2899a3813b0f4ceb8b3a14a933b2c326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.838275 17688 tablet_bootstrap.cc:492] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Bootstrap starting.
I20260925 22:00:59.838651 17688 tablet_bootstrap.cc:654] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.838933 17685 tablet_bootstrap.cc:492] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Bootstrap starting.
I20260925 22:00:59.839025 17707 ts_tablet_manager.cc:1918] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:59.839084 17707 log.cc:1196] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/74b701fad8e74b5f9ea7cd1c8c0e37dd
I20260925 22:00:59.839167 17708 ts_tablet_manager.cc:1918] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:59.839232 17708 log.cc:1196] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/74b701fad8e74b5f9ea7cd1c8c0e37dd
I20260925 22:00:59.839323 17685 tablet_bootstrap.cc:654] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.839325 17707 ts_tablet_manager.cc:1939] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 8dd6a102a4714666810a286816c08484: Deleting consensus metadata
I20260925 22:00:59.839478 17708 ts_tablet_manager.cc:1939] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 83754f2353cb49d6b6b7f92c8950bf50: Deleting consensus metadata
I20260925 22:00:59.839716 17161 catalog_manager.cc:4841] TS 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835): tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd (table test-table [id=c4eeb1ffe67b43dc95e249c5ef9a40cc]) successfully deleted
I20260925 22:00:59.839893 17685 tablet_bootstrap.cc:492] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: No bootstrap required, opened a new log
I20260925 22:00:59.839932 17685 ts_tablet_manager.cc:1397] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:59.840056 17685 raft_consensus.cc:348] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.840126 17685 raft_consensus.cc:374] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.840162 17685 raft_consensus.cc:729] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83754f2353cb49d6b6b7f92c8950bf50, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.840204 17685 consensus_queue.cc:260] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.840260 17709 ts_tablet_manager.cc:1905] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:00:59.840282 17685 ts_tablet_manager.cc:1428] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.840902 17451 tablet_service.cc:1467] Processing CreateTablet for tablet 2899a3813b0f4ceb8b3a14a933b2c326 (DEFAULT_TABLE table=test-table [id=b7ecdad01bb94e13b74934902ff827d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:00:59.840999 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2899a3813b0f4ceb8b3a14a933b2c326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:00:59.841812 17709 ts_tablet_manager.cc:1918] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:00:59.841871 17709 log.cc:1196] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/74b701fad8e74b5f9ea7cd1c8c0e37dd
I20260925 22:00:59.842151 17709 ts_tablet_manager.cc:1939] T 74b701fad8e74b5f9ea7cd1c8c0e37dd P 477f8b24c46f4e33996c53ca48f402f8: Deleting consensus metadata
I20260925 22:00:59.842239 17686 tablet_bootstrap.cc:492] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Bootstrap starting.
I20260925 22:00:59.842490 17159 catalog_manager.cc:4841] TS 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991): tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd (table test-table [id=c4eeb1ffe67b43dc95e249c5ef9a40cc]) successfully deleted
I20260925 22:00:59.842612 17686 tablet_bootstrap.cc:654] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Neither blocks nor log segments found. Creating new log.
I20260925 22:00:59.842743 17688 tablet_bootstrap.cc:492] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: No bootstrap required, opened a new log
I20260925 22:00:59.842801 17688 ts_tablet_manager.cc:1397] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 22:00:59.842962 17688 raft_consensus.cc:348] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.843030 17688 raft_consensus.cc:374] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.843050 17688 raft_consensus.cc:729] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6a102a4714666810a286816c08484, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.843098 17688 consensus_queue.cc:260] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.843185 17688 ts_tablet_manager.cc:1428] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.843182 17686 tablet_bootstrap.cc:492] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: No bootstrap required, opened a new log
I20260925 22:00:59.843230 17686 ts_tablet_manager.cc:1397] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:00:59.843367 17686 raft_consensus.cc:348] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.843436 17686 raft_consensus.cc:374] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:00:59.843459 17686 raft_consensus.cc:729] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Initialized, Role: FOLLOWER
I20260925 22:00:59.843498 17686 consensus_queue.cc:260] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:00:59.843578 17686 ts_tablet_manager.cc:1428] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:00:59.843744 17159 catalog_manager.cc:4841] TS 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301): tablet 74b701fad8e74b5f9ea7cd1c8c0e37dd (table test-table [id=c4eeb1ffe67b43dc95e249c5ef9a40cc]) successfully deleted
W20260925 22:00:59.872062 17378 tablet.cc:2369] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:00:59.954735 17658 tablet.cc:2369] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:01:00.093329 17693 raft_consensus.cc:482] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:01:00.093442 17693 raft_consensus.cc:504] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.093761 17693 leader_election.cc:288] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.097126 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.094150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:45239 (local address 127.9.112.66:42991)
0925 22:01:00.094200 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.094203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.094296 (+ 93us) server_negotiation.cc:407] Connection header received
0925 22:01:00.094378 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.094379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.094389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.094436 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:00.094592 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.095092 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.096288 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.096784 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.096854 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.096862 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.097036 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.097063 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:01:00.097172 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.094002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42991 (local address 127.9.112.65:45239)
0925 22:01:00.094188 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.094202 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.094302 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.094452 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.094453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.094455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:00.094519 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.094523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.095197 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.095198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.096173 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.096176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.096880 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.096881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.096957 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.096962 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.097032 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.097078 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":65,"threads_started":1}
I20260925 22:01:00.097445 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.094168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:46301 (local address 127.9.112.65:49201)
0925 22:01:00.094246 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.094255 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.094308 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.094516 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.094516 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.094518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:00.094587 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.094589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.095436 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.095437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.096493 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.096496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.097160 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.097161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.097231 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.097236 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.097332 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.097372 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"thread_start_us":37,"threads_started":1}
I20260925 22:01:00.097447 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.094284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:49201 (local address 127.9.112.67:46301)
0925 22:01:00.094332 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.094334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.094344 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:01:00.094428 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.094429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.094442 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.094506 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:00.094763 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.095302 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.096583 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.097082 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.097144 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.097152 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.097348 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.097390 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:01:00.097425 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2899a3813b0f4ceb8b3a14a933b2c326" candidate_uuid: "8dd6a102a4714666810a286816c08484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477f8b24c46f4e33996c53ca48f402f8" is_pre_election: true
I20260925 22:01:00.097519 17471 raft_consensus.cc:2454] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd6a102a4714666810a286816c08484 in term 0.
I20260925 22:01:00.097710 17263 leader_election.cc:302] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 8dd6a102a4714666810a286816c08484; no voters:
I20260925 22:01:00.097767 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2899a3813b0f4ceb8b3a14a933b2c326" candidate_uuid: "8dd6a102a4714666810a286816c08484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50" is_pre_election: true
I20260925 22:01:00.097879 17611 raft_consensus.cc:2454] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd6a102a4714666810a286816c08484 in term 0.
I20260925 22:01:00.097892 17693 raft_consensus.cc:2781] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:01:00.097927 17693 raft_consensus.cc:482] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:01:00.097951 17693 raft_consensus.cc:3037] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.098639 17693 raft_consensus.cc:504] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.098744 17693 leader_election.cc:288] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [CANDIDATE]: Term 1 election: Requested vote from peers 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.098898 17471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2899a3813b0f4ceb8b3a14a933b2c326" candidate_uuid: "8dd6a102a4714666810a286816c08484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477f8b24c46f4e33996c53ca48f402f8"
I20260925 22:01:00.098923 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2899a3813b0f4ceb8b3a14a933b2c326" candidate_uuid: "8dd6a102a4714666810a286816c08484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50"
I20260925 22:01:00.098977 17471 raft_consensus.cc:3037] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.098984 17611 raft_consensus.cc:3037] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.099701 17471 raft_consensus.cc:2454] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd6a102a4714666810a286816c08484 in term 1.
I20260925 22:01:00.099710 17611 raft_consensus.cc:2454] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd6a102a4714666810a286816c08484 in term 1.
I20260925 22:01:00.099911 17263 leader_election.cc:302] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 8dd6a102a4714666810a286816c08484; no voters:
I20260925 22:01:00.100000 17693 raft_consensus.cc:2781] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:00.100103 17693 raft_consensus.cc:686] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 1 LEADER]: Becoming Leader. State: Replica: 8dd6a102a4714666810a286816c08484, State: Running, Role: LEADER
I20260925 22:01:00.100190 17693 consensus_queue.cc:237] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.100878 17174 catalog_manager.cc:5495] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dd6a102a4714666810a286816c08484 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "8dd6a102a4714666810a286816c08484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } health_report { overall_health: UNKNOWN } } }
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373659, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.126@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:01:00.160739 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.153302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52944 (local address 127.9.112.126:33253)
0925 22:01:00.153360 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.153361 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.153373 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:01:00.153449 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.153450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.153458 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.153496 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:00.154016 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.154520 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.155274 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.155470 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.155541 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.157822 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:00.157824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:00.157824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:00.157830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:00.159038 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:00.159460 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:00.159462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:00.159462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:00.159584 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:00.160051 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:00.160053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:00.160054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:00.160258 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:00.160331 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.160592 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.160677 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:01:00.160930 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.153158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33253 (local address 127.0.0.1:52944)
0925 22:01:00.153302 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.153318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.153391 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.153515 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.153516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.153853 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:00.153901 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.153903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.154599 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.154600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.155177 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.155179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.155585 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.155586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.155652 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.156319 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:00.156324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:00.157715 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:00.159126 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:00.159133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:00.159136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:00.159341 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:00.159791 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:00.159792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:00.159792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:00.159882 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:00.160418 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:00.160424 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:00.160481 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.160555 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.160771 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":71,"thread_start_us":68,"threads_started":1}
I20260925 22:01:00.163317 17174 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:52944:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 22:01:00.163517 17174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:01:00.165086 17311 tablet_service.cc:1467] Processing CreateTablet for tablet 63d5144c8a3945128087c2d98b8bb388 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=006ee4df628a417a8c12209a354fb85e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:01:00.165119 17451 tablet_service.cc:1467] Processing CreateTablet for tablet 63d5144c8a3945128087c2d98b8bb388 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=006ee4df628a417a8c12209a354fb85e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:01:00.165138 17591 tablet_service.cc:1467] Processing CreateTablet for tablet 63d5144c8a3945128087c2d98b8bb388 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=006ee4df628a417a8c12209a354fb85e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 22:01:00.165270 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d5144c8a3945128087c2d98b8bb388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.165269 17311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d5144c8a3945128087c2d98b8bb388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.165269 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d5144c8a3945128087c2d98b8bb388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.166445 17686 tablet_bootstrap.cc:492] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: Bootstrap starting.
I20260925 22:01:00.166718 17685 tablet_bootstrap.cc:492] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: Bootstrap starting.
I20260925 22:01:00.166832 17686 tablet_bootstrap.cc:654] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.167003 17685 tablet_bootstrap.cc:654] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.167217 17688 tablet_bootstrap.cc:492] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: Bootstrap starting.
I20260925 22:01:00.167443 17686 tablet_bootstrap.cc:492] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: No bootstrap required, opened a new log
I20260925 22:01:00.167493 17686 ts_tablet_manager.cc:1397] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:00.167614 17688 tablet_bootstrap.cc:654] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.167642 17686 raft_consensus.cc:348] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.167725 17686 raft_consensus.cc:374] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.167753 17686 raft_consensus.cc:729] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.167815 17686 consensus_queue.cc:260] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.167917 17686 ts_tablet_manager.cc:1428] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:00.167979 17692 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.168087 17692 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.168202 17685 tablet_bootstrap.cc:492] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: No bootstrap required, opened a new log
I20260925 22:01:00.168246 17685 ts_tablet_manager.cc:1397] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260925 22:01:00.168308 17688 tablet_bootstrap.cc:492] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: No bootstrap required, opened a new log
I20260925 22:01:00.168375 17688 ts_tablet_manager.cc:1397] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:00.168373 17685 raft_consensus.cc:348] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.168437 17685 raft_consensus.cc:374] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.168457 17685 raft_consensus.cc:729] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83754f2353cb49d6b6b7f92c8950bf50, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.168499 17685 consensus_queue.cc:260] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.168509 17688 raft_consensus.cc:348] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.168568 17688 raft_consensus.cc:374] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.168586 17688 raft_consensus.cc:729] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6a102a4714666810a286816c08484, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.168581 17685 ts_tablet_manager.cc:1428] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:00.168627 17688 consensus_queue.cc:260] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.168704 17688 ts_tablet_manager.cc:1428] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:00.168670 17695 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.168733 17695 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.168802 17693 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.168857 17693 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.271404 17692 raft_consensus.cc:482] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:01:00.271518 17692 raft_consensus.cc:504] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.271862 17692 leader_election.cc:288] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.274801 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.272153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:56207 (local address 127.9.112.67:46301)
0925 22:01:00.272199 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.272201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.272277 (+ 76us) server_negotiation.cc:407] Connection header received
0925 22:01:00.272363 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.272364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.272374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.272432 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:00.272554 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.273035 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.274081 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.274473 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.274539 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.274548 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.274684 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.274723 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:01:00.274818 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.272074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:46301 (local address 127.9.112.66:56207)
0925 22:01:00.272204 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.272214 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.272281 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.272439 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.272439 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.272441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:00.272498 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.272500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.273116 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.273117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.273996 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.273999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.274550 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.274551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.274616 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.274620 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.274672 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.274716 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260925 22:01:00.274947 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.272093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37835 (local address 127.9.112.66:55161)
0925 22:01:00.272377 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.272389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.272460 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.272586 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.272586 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.272588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:00.272660 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.272662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.273241 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.273241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.274031 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.274033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.274546 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.274546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.274695 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.274701 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.274771 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.274868 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":55,"threads_started":1}
I20260925 22:01:00.274940 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.272156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:55161 (local address 127.9.112.65:37835)
0925 22:01:00.272218 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.272220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.272446 (+ 226us) server_negotiation.cc:407] Connection header received
0925 22:01:00.272529 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.272530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.272539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.272579 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:00.272717 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.273172 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.274107 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.274475 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.274537 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.274546 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.274795 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.274845 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:01:00.275067 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5144c8a3945128087c2d98b8bb388" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50" is_pre_election: true
I20260925 22:01:00.275161 17611 raft_consensus.cc:2454] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 0.
I20260925 22:01:00.275362 17403 leader_election.cc:302] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 83754f2353cb49d6b6b7f92c8950bf50; no voters:
I20260925 22:01:00.275391 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5144c8a3945128087c2d98b8bb388" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484" is_pre_election: true
I20260925 22:01:00.275457 17331 raft_consensus.cc:2454] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 0.
I20260925 22:01:00.275512 17692 raft_consensus.cc:2781] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:01:00.275553 17692 raft_consensus.cc:482] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:01:00.275578 17692 raft_consensus.cc:3037] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.276232 17692 raft_consensus.cc:504] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.276400 17692 leader_election.cc:288] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 election: Requested vote from peers 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.276590 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5144c8a3945128087c2d98b8bb388" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50"
I20260925 22:01:00.276615 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5144c8a3945128087c2d98b8bb388" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484"
I20260925 22:01:00.276670 17611 raft_consensus.cc:3037] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.276677 17331 raft_consensus.cc:3037] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.277423 17611 raft_consensus.cc:2454] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 1.
I20260925 22:01:00.277436 17331 raft_consensus.cc:2454] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 1.
I20260925 22:01:00.277607 17405 leader_election.cc:302] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 8dd6a102a4714666810a286816c08484; no voters:
I20260925 22:01:00.277726 17692 raft_consensus.cc:2781] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:00.277930 17692 raft_consensus.cc:686] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 LEADER]: Becoming Leader. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Running, Role: LEADER
I20260925 22:01:00.278009 17692 consensus_queue.cc:237] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.278389 17728 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: TxnStatusTablet state changed. Reason: New leader 477f8b24c46f4e33996c53ca48f402f8. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.278487 17728 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:01:00.278654 17730 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 22:01:00.278779 17730 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 22:01:00.278790 17173 catalog_manager.cc:5495] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } health_report { overall_health: UNKNOWN } } }
I20260925 22:01:00.287556 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.285181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.9.112.66:42991)
0925 22:01:00.285258 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.285261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.285281 (+ 20us) server_negotiation.cc:407] Connection header received
0925 22:01:00.285314 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.285315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.285335 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.285393 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:00.285527 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.286036 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.286785 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.286990 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.287044 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.287226 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.287335 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.287399 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.287450 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.287471 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:01:00.287569 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.285099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42991 (local address 127.0.0.1:49106)
0925 22:01:00.285172 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.285184 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.285229 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.285405 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.285406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.285407 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:01:00.285463 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.285466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.286131 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.286132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.286700 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.286702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.287057 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.287058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.287126 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.287142 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.287397 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.287404 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.287447 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.287502 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:01:00.288882 17331 raft_consensus.cc:1261] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Refusing update from remote peer 477f8b24c46f4e33996c53ca48f402f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.288899 17611 raft_consensus.cc:1261] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Refusing update from remote peer 477f8b24c46f4e33996c53ca48f402f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.289148 17692 consensus_queue.cc:1035] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.289283 17728 consensus_queue.cc:1035] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.290446 17695 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: TxnStatusTablet state changed. Reason: New leader 477f8b24c46f4e33996c53ca48f402f8. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.290548 17695 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.290477 17693 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: TxnStatusTablet state changed. Reason: New leader 477f8b24c46f4e33996c53ca48f402f8. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.290580 17693 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.291320 17698 mvcc.cc:205] Tried to move back new op lower bound from 7333370512541016064 to 7333370512499261440. Current Snapshot: MvccSnapshot[applied={T|T < 7333370512541016064}]
I20260925 22:01:00.291415 17695 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.291481 17695 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 83754f2353cb49d6b6b7f92c8950bf50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.291450 17728 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.291523 17728 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:01:00.291560 17728 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.291669 17728 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 477f8b24c46f4e33996c53ca48f402f8: This TxnStatusTablet replica's current role is: LEADER
I20260925 22:01:00.292033 17693 tablet_replica.cc:439] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } } }
I20260925 22:01:00.292126 17693 tablet_replica.cc:442] T 63d5144c8a3945128087c2d98b8bb388 P 8dd6a102a4714666810a286816c08484: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 22:01:00.296458 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.293422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.9.112.65:37835)
0925 22:01:00.293473 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.293475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.293485 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:01:00.293548 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.293549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.293571 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.293644 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:00.293735 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.294755 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.295555 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.295875 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.295946 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.296076 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.296219 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.296291 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.296307 (+ 16us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.296329 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:01:00.302558 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.299202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40895 (local address 127.9.112.66:42991)
0925 22:01:00.299246 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.299248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.299401 (+ 153us) server_negotiation.cc:407] Connection header received
0925 22:01:00.299429 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.299430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.299439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.299494 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:00.299723 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.300399 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.301657 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.301939 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.302015 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.302280 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.302317 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.302374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.302464 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.302491 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:01:00.302615 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.299110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42991 (local address 127.9.112.65:40895)
0925 22:01:00.299263 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.299274 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.299339 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.299502 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.299503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.299505 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:01:00.299584 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.299585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.300518 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.300519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.301528 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.301531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.302057 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.302058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.302135 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.302162 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.302385 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.302387 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.302454 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.302505 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260925 22:01:00.305085 17611 raft_consensus.cc:1261] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Refusing update from remote peer 8dd6a102a4714666810a286816c08484: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.305140 17471 raft_consensus.cc:1261] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Refusing update from remote peer 8dd6a102a4714666810a286816c08484: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.305339 17693 consensus_queue.cc:1035] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.305496 17715 consensus_queue.cc:1035] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.314810 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.311506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:47101 (local address 127.9.112.65:37835)
0925 22:01:00.311546 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.311548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.311745 (+ 197us) server_negotiation.cc:407] Connection header received
0925 22:01:00.311991 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.311993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.312004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.312068 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:00.312186 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.312996 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.313995 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.314306 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.314375 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.314505 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.314564 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.314614 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.314725 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.314752 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 22:01:00.314855 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.311460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37835 (local address 127.9.112.66:47101)
0925 22:01:00.311676 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.311687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.311773 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.312071 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.312072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.312074 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:01:00.312139 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.312140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.313081 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.313081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.313936 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.313939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.314374 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.314374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.314422 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.314437 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.314642 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.314644 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.314690 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.314744 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
I20260925 22:01:00.330665 17173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:00.330777 17173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:00.331393 17173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Sending DeleteTablet for 3 replicas of tablet 2899a3813b0f4ceb8b3a14a933b2c326
I20260925 22:01:00.331643 17311 tablet_service.cc:1514] Processing DeleteTablet for tablet 2899a3813b0f4ceb8b3a14a933b2c326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:32994
I20260925 22:01:00.331836 17591 tablet_service.cc:1514] Processing DeleteTablet for tablet 2899a3813b0f4ceb8b3a14a933b2c326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:36742
I20260925 22:01:00.331907 17707 tablet_replica.cc:330] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: stopping tablet replica
I20260925 22:01:00.331972 17707 raft_consensus.cc:2229] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:01:00.331921 17708 tablet_replica.cc:330] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: stopping tablet replica
I20260925 22:01:00.332051 17708 raft_consensus.cc:2229] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:00.332070 17707 raft_consensus.cc:2258] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.332093 17708 raft_consensus.cc:2258] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.332511 17708 ts_tablet_manager.cc:1905] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.332593 17707 ts_tablet_manager.cc:1905] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.333865 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet 2899a3813b0f4ceb8b3a14a933b2c326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49090
I20260925 22:01:00.334283 17708 ts_tablet_manager.cc:1918] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:01:00.334352 17708 log.cc:1196] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/2899a3813b0f4ceb8b3a14a933b2c326
I20260925 22:01:00.334448 17707 ts_tablet_manager.cc:1918] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:01:00.334492 17707 log.cc:1196] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/2899a3813b0f4ceb8b3a14a933b2c326
I20260925 22:01:00.334545 17708 ts_tablet_manager.cc:1939] T 2899a3813b0f4ceb8b3a14a933b2c326 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting consensus metadata
I20260925 22:01:00.334666 17707 ts_tablet_manager.cc:1939] T 2899a3813b0f4ceb8b3a14a933b2c326 P 8dd6a102a4714666810a286816c08484: Deleting consensus metadata
I20260925 22:01:00.334791 17709 tablet_replica.cc:330] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: stopping tablet replica
I20260925 22:01:00.334863 17709 raft_consensus.cc:2229] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:00.334900 17709 raft_consensus.cc:2258] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.335412 17709 ts_tablet_manager.cc:1905] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.335750 17161 catalog_manager.cc:4841] TS 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835): tablet 2899a3813b0f4ceb8b3a14a933b2c326 (table test-table [id=b7ecdad01bb94e13b74934902ff827d6]) successfully deleted
I20260925 22:01:00.337616 17159 catalog_manager.cc:4841] TS 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301): tablet 2899a3813b0f4ceb8b3a14a933b2c326 (table test-table [id=b7ecdad01bb94e13b74934902ff827d6]) successfully deleted
I20260925 22:01:00.337952 17709 ts_tablet_manager.cc:1918] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:01:00.338021 17709 log.cc:1196] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/2899a3813b0f4ceb8b3a14a933b2c326
I20260925 22:01:00.338618 17709 ts_tablet_manager.cc:1939] T 2899a3813b0f4ceb8b3a14a933b2c326 P 477f8b24c46f4e33996c53ca48f402f8: Deleting consensus metadata
I20260925 22:01:00.339084 17159 catalog_manager.cc:4841] TS 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991): tablet 2899a3813b0f4ceb8b3a14a933b2c326 (table test-table [id=b7ecdad01bb94e13b74934902ff827d6]) successfully deleted
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:01:00.358350 17224 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:00.352999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.9.112.126:33253)
0925 22:01:00.353051 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.353053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.353176 (+ 123us) server_negotiation.cc:407] Connection header received
0925 22:01:00.353245 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.353246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.353257 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.353305 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:00.353852 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.354345 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.355051 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.355241 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.355297 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.358296 (+ 2999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52946: BlockingRecv error: recv got EOF from 127.0.0.1:52946 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM
I20260925 22:01:00.366149 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.359598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.9.112.126:33253)
0925 22:01:00.359658 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.359660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.359667 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:01:00.359692 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.359693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.359701 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.359746 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:00.360217 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.360801 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.361487 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.361691 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.361745 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.363820 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:00.363822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:00.363822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:00.363830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:00.365138 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:00.365450 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:00.365451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:00.365452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:00.365508 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:00.365672 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:00.365673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:00.365673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:00.365790 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:00.365875 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.365973 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.366051 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 22:01:00.367597 17173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:01:00.367756 17173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:01:00.368949 17311 tablet_service.cc:1467] Processing CreateTablet for tablet ecda00fe9bfd44d79a9f00bae4ad1274 (DEFAULT_TABLE table=test-table [id=1114a3775b334487ab3502d535f4fadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:00.369091 17311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecda00fe9bfd44d79a9f00bae4ad1274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.369081 17451 tablet_service.cc:1467] Processing CreateTablet for tablet ecda00fe9bfd44d79a9f00bae4ad1274 (DEFAULT_TABLE table=test-table [id=1114a3775b334487ab3502d535f4fadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:00.369184 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecda00fe9bfd44d79a9f00bae4ad1274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.370496 17688 tablet_bootstrap.cc:492] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Bootstrap starting.
I20260925 22:01:00.370481 17591 tablet_service.cc:1467] Processing CreateTablet for tablet ecda00fe9bfd44d79a9f00bae4ad1274 (DEFAULT_TABLE table=test-table [id=1114a3775b334487ab3502d535f4fadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:00.370594 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecda00fe9bfd44d79a9f00bae4ad1274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.371025 17688 tablet_bootstrap.cc:654] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.372520 17686 tablet_bootstrap.cc:492] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Bootstrap starting.
I20260925 22:01:00.372833 17686 tablet_bootstrap.cc:654] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.373118 17685 tablet_bootstrap.cc:492] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Bootstrap starting.
I20260925 22:01:00.373433 17685 tablet_bootstrap.cc:654] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.374241 17685 tablet_bootstrap.cc:492] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: No bootstrap required, opened a new log
I20260925 22:01:00.374380 17685 ts_tablet_manager.cc:1397] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:00.374534 17685 raft_consensus.cc:348] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.374606 17685 raft_consensus.cc:374] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.374625 17685 raft_consensus.cc:729] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83754f2353cb49d6b6b7f92c8950bf50, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.374810 17685 consensus_queue.cc:260] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.374969 17685 ts_tablet_manager.cc:1428] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:00.375357 17686 tablet_bootstrap.cc:492] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: No bootstrap required, opened a new log
I20260925 22:01:00.375411 17686 ts_tablet_manager.cc:1397] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 22:01:00.375551 17686 raft_consensus.cc:348] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.375618 17686 raft_consensus.cc:374] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.375635 17686 raft_consensus.cc:729] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.375679 17686 consensus_queue.cc:260] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.375777 17686 ts_tablet_manager.cc:1428] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:00.376821 17688 tablet_bootstrap.cc:492] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: No bootstrap required, opened a new log
I20260925 22:01:00.376875 17688 ts_tablet_manager.cc:1397] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 22:01:00.376999 17688 raft_consensus.cc:348] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.377090 17688 raft_consensus.cc:374] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.377115 17688 raft_consensus.cc:729] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6a102a4714666810a286816c08484, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.377153 17688 consensus_queue.cc:260] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.377233 17688 ts_tablet_manager.cc:1428] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:01:00.382714 17692 raft_consensus.cc:482] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:01:00.382812 17692 raft_consensus.cc:504] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.383070 17692 leader_election.cc:288] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.383280 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecda00fe9bfd44d79a9f00bae4ad1274" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484" is_pre_election: true
I20260925 22:01:00.383296 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecda00fe9bfd44d79a9f00bae4ad1274" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50" is_pre_election: true
I20260925 22:01:00.383368 17331 raft_consensus.cc:2454] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 0.
I20260925 22:01:00.383391 17611 raft_consensus.cc:2454] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 0.
I20260925 22:01:00.383523 17405 leader_election.cc:302] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 83754f2353cb49d6b6b7f92c8950bf50, 8dd6a102a4714666810a286816c08484; no voters:
I20260925 22:01:00.383602 17692 raft_consensus.cc:2781] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:01:00.383625 17692 raft_consensus.cc:482] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:01:00.383642 17692 raft_consensus.cc:3037] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.384533 17692 raft_consensus.cc:504] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.384661 17692 leader_election.cc:288] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 election: Requested vote from peers 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.384856 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecda00fe9bfd44d79a9f00bae4ad1274" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50"
I20260925 22:01:00.384877 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecda00fe9bfd44d79a9f00bae4ad1274" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484"
I20260925 22:01:00.384936 17611 raft_consensus.cc:3037] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.384938 17331 raft_consensus.cc:3037] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.385843 17331 raft_consensus.cc:2454] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 1.
I20260925 22:01:00.385859 17611 raft_consensus.cc:2454] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 1.
I20260925 22:01:00.386031 17405 leader_election.cc:302] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 8dd6a102a4714666810a286816c08484; no voters:
I20260925 22:01:00.386139 17692 raft_consensus.cc:2781] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:00.386193 17692 raft_consensus.cc:686] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 LEADER]: Becoming Leader. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Running, Role: LEADER
I20260925 22:01:00.386240 17692 consensus_queue.cc:237] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.386862 17173 catalog_manager.cc:5495] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } health_report { overall_health: UNKNOWN } } }
I20260925 22:01:00.402228 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.399175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.9.112.66:42991)
0925 22:01:00.399216 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.399218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.399263 (+ 45us) server_negotiation.cc:407] Connection header received
0925 22:01:00.399315 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.399317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.399341 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.399385 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:00.399590 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.400312 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.401191 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.401502 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.401573 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.401890 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.401926 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.401986 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.402104 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.402130 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:01:00.403100 17331 raft_consensus.cc:1261] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Refusing update from remote peer 477f8b24c46f4e33996c53ca48f402f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.403102 17611 raft_consensus.cc:1261] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Refusing update from remote peer 477f8b24c46f4e33996c53ca48f402f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.403330 17692 consensus_queue.cc:1035] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.403458 17735 consensus_queue.cc:1035] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.407703 17173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:00.407804 17173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:00.408535 17173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Sending DeleteTablet for 3 replicas of tablet ecda00fe9bfd44d79a9f00bae4ad1274
I20260925 22:01:00.408794 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet ecda00fe9bfd44d79a9f00bae4ad1274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49090
I20260925 22:01:00.408905 17591 tablet_service.cc:1514] Processing DeleteTablet for tablet ecda00fe9bfd44d79a9f00bae4ad1274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:36742
I20260925 22:01:00.408994 17709 tablet_replica.cc:330] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: stopping tablet replica
I20260925 22:01:00.409044 17709 raft_consensus.cc:2229] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:01:00.409152 17709 raft_consensus.cc:2258] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.409235 17708 tablet_replica.cc:330] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: stopping tablet replica
I20260925 22:01:00.409297 17708 raft_consensus.cc:2229] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:00.409344 17708 raft_consensus.cc:2258] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.409621 17709 ts_tablet_manager.cc:1905] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.409832 17311 tablet_service.cc:1514] Processing DeleteTablet for tablet ecda00fe9bfd44d79a9f00bae4ad1274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:32994
I20260925 22:01:00.409919 17707 tablet_replica.cc:330] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: stopping tablet replica
I20260925 22:01:00.409958 17707 raft_consensus.cc:2229] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:00.409997 17707 raft_consensus.cc:2258] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.410300 17173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:01:00.410456 17173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:01:00.411357 17709 ts_tablet_manager.cc:1918] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:01:00.411412 17709 log.cc:1196] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/ecda00fe9bfd44d79a9f00bae4ad1274
I20260925 22:01:00.411602 17709 ts_tablet_manager.cc:1939] T ecda00fe9bfd44d79a9f00bae4ad1274 P 477f8b24c46f4e33996c53ca48f402f8: Deleting consensus metadata
I20260925 22:01:00.411706 17591 tablet_service.cc:1467] Processing CreateTablet for tablet 8d59440509964199980a88a89e92d2a6 (DEFAULT_TABLE table=test-table [id=af6600cb05b448faacf27019f0ae0d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:00.411824 17591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d59440509964199980a88a89e92d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.412518 17707 ts_tablet_manager.cc:1905] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.413184 17685 tablet_bootstrap.cc:492] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Bootstrap starting.
I20260925 22:01:00.413492 17685 tablet_bootstrap.cc:654] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.413942 17707 ts_tablet_manager.cc:1918] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:01:00.413965 17451 tablet_service.cc:1467] Processing CreateTablet for tablet 8d59440509964199980a88a89e92d2a6 (DEFAULT_TABLE table=test-table [id=af6600cb05b448faacf27019f0ae0d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:00.413990 17707 log.cc:1196] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/ecda00fe9bfd44d79a9f00bae4ad1274
I20260925 22:01:00.414055 17451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d59440509964199980a88a89e92d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.414168 17707 ts_tablet_manager.cc:1939] T ecda00fe9bfd44d79a9f00bae4ad1274 P 8dd6a102a4714666810a286816c08484: Deleting consensus metadata
I20260925 22:01:00.415261 17686 tablet_bootstrap.cc:492] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Bootstrap starting.
I20260925 22:01:00.415300 17159 catalog_manager.cc:4841] TS 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991): tablet ecda00fe9bfd44d79a9f00bae4ad1274 (table test-table [id=1114a3775b334487ab3502d535f4fadb]) successfully deleted
I20260925 22:01:00.415565 17161 catalog_manager.cc:4841] TS 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835): tablet ecda00fe9bfd44d79a9f00bae4ad1274 (table test-table [id=1114a3775b334487ab3502d535f4fadb]) successfully deleted
I20260925 22:01:00.415694 17686 tablet_bootstrap.cc:654] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.415818 17311 tablet_service.cc:1467] Processing CreateTablet for tablet 8d59440509964199980a88a89e92d2a6 (DEFAULT_TABLE table=test-table [id=af6600cb05b448faacf27019f0ae0d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:00.415910 17311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d59440509964199980a88a89e92d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:00.416445 17686 tablet_bootstrap.cc:492] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: No bootstrap required, opened a new log
I20260925 22:01:00.416491 17686 ts_tablet_manager.cc:1397] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:00.416626 17686 raft_consensus.cc:348] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.416694 17686 raft_consensus.cc:374] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.416711 17686 raft_consensus.cc:729] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.416752 17686 consensus_queue.cc:260] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.416834 17686 ts_tablet_manager.cc:1428] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:00.417085 17688 tablet_bootstrap.cc:492] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484: Bootstrap starting.
I20260925 22:01:00.417414 17688 tablet_bootstrap.cc:654] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:00.417968 17688 tablet_bootstrap.cc:492] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484: No bootstrap required, opened a new log
I20260925 22:01:00.418028 17688 ts_tablet_manager.cc:1397] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:00.418174 17688 raft_consensus.cc:348] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.418236 17688 raft_consensus.cc:374] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.418274 17688 raft_consensus.cc:729] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6a102a4714666810a286816c08484, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.418313 17688 consensus_queue.cc:260] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.418412 17688 ts_tablet_manager.cc:1428] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 22:01:00.419224 17708 ts_tablet_manager.cc:1905] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.419404 17685 tablet_bootstrap.cc:492] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: No bootstrap required, opened a new log
I20260925 22:01:00.419442 17685 ts_tablet_manager.cc:1397] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 22:01:00.419538 17685 raft_consensus.cc:348] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.419593 17685 raft_consensus.cc:374] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:00.419608 17685 raft_consensus.cc:729] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83754f2353cb49d6b6b7f92c8950bf50, State: Initialized, Role: FOLLOWER
I20260925 22:01:00.419646 17685 consensus_queue.cc:260] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.419734 17685 ts_tablet_manager.cc:1428] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:00.420805 17708 ts_tablet_manager.cc:1918] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:01:00.420848 17708 log.cc:1196] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/ecda00fe9bfd44d79a9f00bae4ad1274
I20260925 22:01:00.420980 17708 ts_tablet_manager.cc:1939] T ecda00fe9bfd44d79a9f00bae4ad1274 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting consensus metadata
I20260925 22:01:00.421762 17159 catalog_manager.cc:4841] TS 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301): tablet ecda00fe9bfd44d79a9f00bae4ad1274 (table test-table [id=1114a3775b334487ab3502d535f4fadb]) successfully deleted
I20260925 22:01:00.649667 17735 raft_consensus.cc:482] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:01:00.649772 17735 raft_consensus.cc:504] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.649960 17735 leader_election.cc:288] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.650182 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59440509964199980a88a89e92d2a6" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50" is_pre_election: true
I20260925 22:01:00.650265 17611 raft_consensus.cc:2454] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 0.
I20260925 22:01:00.650197 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59440509964199980a88a89e92d2a6" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484" is_pre_election: true
I20260925 22:01:00.650359 17331 raft_consensus.cc:2454] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 0.
I20260925 22:01:00.650421 17403 leader_election.cc:302] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 83754f2353cb49d6b6b7f92c8950bf50; no voters:
I20260925 22:01:00.650521 17735 raft_consensus.cc:2781] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:01:00.650571 17735 raft_consensus.cc:482] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:01:00.650591 17735 raft_consensus.cc:3037] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.651295 17735 raft_consensus.cc:504] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.651437 17735 leader_election.cc:288] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 election: Requested vote from peers 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835), 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301)
I20260925 22:01:00.651675 17331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59440509964199980a88a89e92d2a6" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd6a102a4714666810a286816c08484"
I20260925 22:01:00.651772 17331 raft_consensus.cc:3037] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.651675 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59440509964199980a88a89e92d2a6" candidate_uuid: "477f8b24c46f4e33996c53ca48f402f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83754f2353cb49d6b6b7f92c8950bf50"
I20260925 22:01:00.651814 17611 raft_consensus.cc:3037] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:00.652622 17611 raft_consensus.cc:2454] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 1.
I20260925 22:01:00.652644 17331 raft_consensus.cc:2454] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477f8b24c46f4e33996c53ca48f402f8 in term 1.
I20260925 22:01:00.652812 17403 leader_election.cc:302] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477f8b24c46f4e33996c53ca48f402f8, 83754f2353cb49d6b6b7f92c8950bf50; no voters:
I20260925 22:01:00.652926 17735 raft_consensus.cc:2781] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:00.652976 17735 raft_consensus.cc:686] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 LEADER]: Becoming Leader. State: Replica: 477f8b24c46f4e33996c53ca48f402f8, State: Running, Role: LEADER
I20260925 22:01:00.653025 17735 consensus_queue.cc:237] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } }
I20260925 22:01:00.653640 17173 catalog_manager.cc:5495] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "477f8b24c46f4e33996c53ca48f402f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477f8b24c46f4e33996c53ca48f402f8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 } health_report { overall_health: UNKNOWN } } }
I20260925 22:01:00.709860 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.707682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:48843 (local address 127.9.112.66:42991)
0925 22:01:00.707719 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:00.707721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:00.707750 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:01:00.707801 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:00.707802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:00.707811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:00.707862 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:00.707947 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.708477 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.709216 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.709415 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.709474 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.709621 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.709647 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.709686 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:00.709748 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 22:01:00.709767 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":16}
I20260925 22:01:00.709867 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.707638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42991 (local address 127.9.112.66:48843)
0925 22:01:00.707697 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.707710 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:00.707740 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:00.707863 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:00.707863 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:00.707864 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 22:01:00.707912 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.707913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.708569 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.708570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.709123 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:00.709125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:00.709504 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:00.709505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:00.709551 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:00.709557 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 22:01:00.709710 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 22:01:00.709712 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:01:00.709736 (+ 24us) client_negotiation.cc:240] Negotiation successful
0925 22:01:00.709785 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260925 22:01:00.711840 17611 raft_consensus.cc:1261] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Refusing update from remote peer 477f8b24c46f4e33996c53ca48f402f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.711979 17331 raft_consensus.cc:1261] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Refusing update from remote peer 477f8b24c46f4e33996c53ca48f402f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:00.712098 17735 consensus_queue.cc:1035] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83754f2353cb49d6b6b7f92c8950bf50" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 46301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.712198 17728 consensus_queue.cc:1035] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd6a102a4714666810a286816c08484" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:00.722453 17172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:00.722561 17172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:00.723213 17172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfc93849005b4ea5950e5418175af6bf: Sending DeleteTablet for 3 replicas of tablet 8d59440509964199980a88a89e92d2a6
I20260925 22:01:00.723501 17311 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d59440509964199980a88a89e92d2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:32994
I20260925 22:01:00.723615 17591 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d59440509964199980a88a89e92d2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:36742
I20260925 22:01:00.723639 17707 tablet_replica.cc:330] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484: stopping tablet replica
I20260925 22:01:00.723695 17707 raft_consensus.cc:2229] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:00.723739 17707 raft_consensus.cc:2258] T 8d59440509964199980a88a89e92d2a6 P 8dd6a102a4714666810a286816c08484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.723744 17708 tablet_replica.cc:330] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: stopping tablet replica
I20260925 22:01:00.723815 17708 raft_consensus.cc:2229] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:00.723855 17708 raft_consensus.cc:2258] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.723922 17451 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d59440509964199980a88a89e92d2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:00 UTC) from {username='oryx', principal='oryx/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49090
I20260925 22:01:00.724015 17709 tablet_replica.cc:330] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: stopping tablet replica
I20260925 22:01:00.724063 17709 raft_consensus.cc:2229] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:01:00.724128 17709 raft_consensus.cc:2258] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:00.724182 17708 ts_tablet_manager.cc:1905] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.724243 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 17246
I20260925 22:01:00.724474 17709 ts_tablet_manager.cc:1905] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:00.725839 17708 ts_tablet_manager.cc:1918] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:01:00.725915 17708 log.cc:1196] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/8d59440509964199980a88a89e92d2a6
I20260925 22:01:00.725952 17709 ts_tablet_manager.cc:1918] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 22:01:00.726007 17709 log.cc:1196] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/8d59440509964199980a88a89e92d2a6
I20260925 22:01:00.726127 17708 ts_tablet_manager.cc:1939] T 8d59440509964199980a88a89e92d2a6 P 83754f2353cb49d6b6b7f92c8950bf50: Deleting consensus metadata
I20260925 22:01:00.726214 17709 ts_tablet_manager.cc:1939] T 8d59440509964199980a88a89e92d2a6 P 477f8b24c46f4e33996c53ca48f402f8: Deleting consensus metadata
I20260925 22:01:00.726485 17159 catalog_manager.cc:4841] TS 83754f2353cb49d6b6b7f92c8950bf50 (127.9.112.67:46301): tablet 8d59440509964199980a88a89e92d2a6 (table test-table [id=af6600cb05b448faacf27019f0ae0d2a]) successfully deleted
I20260925 22:01:00.726634 17159 catalog_manager.cc:4841] TS 477f8b24c46f4e33996c53ca48f402f8 (127.9.112.66:42991): tablet 8d59440509964199980a88a89e92d2a6 (table test-table [id=af6600cb05b448faacf27019f0ae0d2a]) successfully deleted
W20260925 22:01:00.731160 17161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:37835 (error 108)
I20260925 22:01:00.731346 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 17386
I20260925 22:01:00.732496 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:00.732243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37835 (local address 127.0.0.1:33010)
0925 22:01:00.732408 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:00.732447 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:37835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":54,"threads_started":1}
W20260925 22:01:00.732585 17161 catalog_manager.cc:4584] TS 8dd6a102a4714666810a286816c08484 (127.9.112.65:37835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d59440509964199980a88a89e92d2a6: Network error: Client connection negotiation failed: client connection to 127.9.112.65:37835: connect: Connection refused (error 111)
I20260925 22:01:00.736703 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 17526
I20260925 22:01:00.741500 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 17142
2026-09-25T22:01:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2804 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17776](info): set up 2 sockets
Sep 25 22:01:00 dist-test-slave-794d krb5kdc[17776](info): commencing operation
krb5kdc: starting...
W20260925 22:01:01.716782 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:01:01 dist-test-slave-794d krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:01:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:01:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:01:01.840515 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:46829
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:36905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:46829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:01.920835 17789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:01.921069 17789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:01.921093 17789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:01.922359 17789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:01:01.922415 17789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:01.922427 17789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:01.922439 17789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:01:01.922451 17789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:01:01.924178 17789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:46829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:46829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:01.924484 17789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:01.924732 17789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:01.929694 17797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:01.929667 17794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:01.929653 17795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:01.929845 17789 server_base.cc:1034] running on GCE node
I20260925 22:01:01.930256 17789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:01.930545 17789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:01.931722 17789 hybrid_clock.cc:648] HybridClock initialized: now 1790373661931681 us; error 53 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T22:01:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1196 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:01:01 dist-test-slave-794d krb5kdc[17802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:01:01 dist-test-slave-794d krb5kdc[17802](info): set up 2 sockets
Sep 25 22:01:01 dist-test-slave-794d krb5kdc[17802](info): commencing operation
krb5kdc: starting...
W20260925 22:01:02.900796 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:01:02 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:01:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:01:03.027235 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45243
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:03.109154 17815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:03.109387 17815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:03.109411 17815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:03.110638 17815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:01:03.110687 17815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:03.110698 17815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:03.110710 17815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:01:03.110720 17815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:01:03.112287 17815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:45243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:03.112547 17815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:03.112739 17815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:03.117744 17821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.117880 17823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:03.117797 17815 server_base.cc:1034] running on GCE node
W20260925 22:01:03.117787 17820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:03.118290 17815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:03.118587 17815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:03.119717 17815 hybrid_clock.cc:648] HybridClock initialized: now 1790373663119705 us; error 26 us; skew 500 ppm
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:03.126287 17815 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:01:03.126844 17815 webserver.cc:460] Webserver started at http://127.9.112.126:44545/ using document root <none> and password file <none>
I20260925 22:01:03.127077 17815 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:03.127139 17815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:03.127259 17815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:03.128429 17815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "dc08e654617d421abc3b032063195d5b"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "50f35367fefbbab792f34c7b4e7000ec"
server_key_iv: "fad0276c12fb58fac166a5e1339f5e89"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.128782 17815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "dc08e654617d421abc3b032063195d5b"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "50f35367fefbbab792f34c7b4e7000ec"
server_key_iv: "fad0276c12fb58fac166a5e1339f5e89"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.130189 17815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:01:03.130991 17830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.131189 17815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:01:03.131276 17815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "dc08e654617d421abc3b032063195d5b"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "50f35367fefbbab792f34c7b4e7000ec"
server_key_iv: "fad0276c12fb58fac166a5e1339f5e89"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.131345 17815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:03.147349 17815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:03.149637 17815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:03.149760 17815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:03.153550 17815 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:45243
I20260925 22:01:03.153611 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:45243 every 8 connection(s)
I20260925 22:01:03.153941 17815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:01:03.154851 17883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:03.157217 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Bootstrap starting.
I20260925 22:01:03.157814 17883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:03.158079 17883 log.cc:824] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:03.158946 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: No bootstrap required, opened a new log
I20260925 22:01:03.160652 17883 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.160815 17883 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:03.160836 17883 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc08e654617d421abc3b032063195d5b, State: Initialized, Role: FOLLOWER
I20260925 22:01:03.160955 17883 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.161031 17883 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:01:03.161062 17883 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:01:03.161108 17883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:03.161715 17883 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.161810 17883 leader_election.cc:302] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc08e654617d421abc3b032063195d5b; no voters:
I20260925 22:01:03.161991 17883 leader_election.cc:288] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:01:03.162082 17886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:03.162478 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 17815
I20260925 22:01:03.162566 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:01:03.162722 9665 external_mini_cluster.cc:1428] Setting key 7ad9794dd4d1909db8d96651645a2ac6
I20260925 22:01:03.162266 17883 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:01:03.162287 17886 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 LEADER]: Becoming Leader. State: Replica: dc08e654617d421abc3b032063195d5b, State: Running, Role: LEADER
I20260925 22:01:03.162973 17886 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.163655 17888 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc08e654617d421abc3b032063195d5b. Latest consensus state: current_term: 1 leader_uuid: "dc08e654617d421abc3b032063195d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } } }
I20260925 22:01:03.163769 17888 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:03.163657 17887 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc08e654617d421abc3b032063195d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } } }
I20260925 22:01:03.163967 17887 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:03.164070 17897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:01:03.164633 17897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:01:03.167667 17897 catalog_manager.cc:1347] Generated new cluster ID: d45e534392a84b0ab7506e5f1b486078
I20260925 22:01:03.167729 17897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:03.174008 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.163210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.9.112.126:45243)
0925 22:01:03.163396 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:03.163405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:03.164171 (+ 766us) server_negotiation.cc:407] Connection header received
0925 22:01:03.164912 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:03.164921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:03.164980 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:03.165067 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:03.165795 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.167180 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.168059 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.168560 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.168655 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.170817 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:03.170821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:03.170821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:03.170833 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:03.172304 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.172647 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.172650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.172650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.172747 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.172943 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.172944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.172944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.173272 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:03.173413 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:03.173685 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 22:01:03.173840 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260925 22:01:03.176599 17897 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:01:03.177093 17897 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:01:03.179978 17897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Generated new TSK 0
I20260925 22:01:03.180176 17897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:01:03.216955 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45243
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:03.322618 17909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:03.322841 17909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:03.322873 17909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:03.324064 17909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:03.324105 17909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:03.324141 17909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:01:03.325639 17909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:03.325882 17909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:03.326093 17909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:03.330988 17914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.331039 17915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.331007 17917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:03.331161 17909 server_base.cc:1034] running on GCE node
I20260925 22:01:03.331485 17909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:03.331732 17909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:03.332890 17909 hybrid_clock.cc:648] HybridClock initialized: now 1790373663332873 us; error 28 us; skew 500 ppm
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:03.335497 17909 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:01:03.335959 17909 webserver.cc:460] Webserver started at http://127.9.112.65:33479/ using document root <none> and password file <none>
I20260925 22:01:03.336140 17909 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:03.336179 17909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:03.336272 17909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:03.337314 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "6dd971fae771464bab48e02d4dd34d36"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "149ff92b6711254051c2719aba0aa55b"
server_key_iv: "d846f033f1852049efe341d3f64f4d7f"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.337604 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "6dd971fae771464bab48e02d4dd34d36"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "149ff92b6711254051c2719aba0aa55b"
server_key_iv: "d846f033f1852049efe341d3f64f4d7f"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.338891 17909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:01:03.339543 17924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.339705 17909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:01:03.339792 17909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "6dd971fae771464bab48e02d4dd34d36"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "149ff92b6711254051c2719aba0aa55b"
server_key_iv: "d846f033f1852049efe341d3f64f4d7f"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.339854 17909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:03.346935 17909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:03.348920 17909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:03.349027 17909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:03.349202 17909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:03.349475 17909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:03.349504 17909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.349524 17909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:03.349535 17909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.354765 17909 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:38181
I20260925 22:01:03.354830 18037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:38181 every 8 connection(s)
I20260925 22:01:03.355105 17909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:03.362986 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 17909
I20260925 22:01:03.363093 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:01:03.363349 9665 external_mini_cluster.cc:1428] Setting key 3eb5d3014d3b0f6a7be85bb090208f71
I20260925 22:01:03.364956 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.355889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:36431 (local address 127.9.112.126:45243)
0925 22:01:03.355947 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:03.355949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:03.356171 (+ 222us) server_negotiation.cc:407] Connection header received
0925 22:01:03.356690 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:03.356691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:03.356703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:03.356773 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:03.357728 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.358280 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.359465 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.359679 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.359732 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.361998 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:03.362000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:03.362000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:03.362010 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:03.363506 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.363947 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.363948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.363948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.364001 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.364225 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.364226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.364226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.364377 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:03.364445 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:03.364711 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 22:01:03.364855 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:01:03.365027 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.355860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:36431)
0925 22:01:03.356026 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:03.356067 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:03.356603 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:03.356826 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:03.356829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:03.357377 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:03.357562 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:03.357566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.358375 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.358376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:03.359364 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:03.359367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.359765 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.359766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:03.359880 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.360416 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:03.360425 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:03.361908 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:03.363579 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:03.363586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:03.363589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:03.363874 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:03.364059 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:03.364060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:03.364060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:03.364164 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:03.364437 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:03.364441 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:03.364562 (+ 121us) client_negotiation.cc:769] Sending connection context
0925 22:01:03.364646 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:01:03.364842 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260925 22:01:03.365799 18038 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45243
I20260925 22:01:03.365890 18038 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:03.366062 18038 heartbeater.cc:511] Master 127.9.112.126:45243 requested a full tablet report, sending...
I20260925 22:01:03.366531 17847 ts_manager.cc:194] Registered new tserver with Master: 6dd971fae771464bab48e02d4dd34d36 (127.9.112.65:38181)
I20260925 22:01:03.367194 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:36431
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:01:03.395489 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45243
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:03.484071 18043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:03.484292 18043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:03.484316 18043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:03.485562 18043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:03.485610 18043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:03.485637 18043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:01:03.487087 18043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:03.487309 18043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:03.487506 18043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:03.492519 18051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.492573 18048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.492576 18049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:03.492784 18043 server_base.cc:1034] running on GCE node
I20260925 22:01:03.492985 18043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:03.493239 18043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:03.494410 18043 hybrid_clock.cc:648] HybridClock initialized: now 1790373663494390 us; error 29 us; skew 500 ppm
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:03.496959 18043 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:01:03.497416 18043 webserver.cc:460] Webserver started at http://127.9.112.66:42325/ using document root <none> and password file <none>
I20260925 22:01:03.497634 18043 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:03.497681 18043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:03.497807 18043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:03.498858 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "9b86aeb5642341d79a592e6bf96b5a82"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "19a5c5eca3f9dc48c2d57e50835c3967"
server_key_iv: "575a004bcffa23bfd1e42fac22c4f45e"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.499186 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "9b86aeb5642341d79a592e6bf96b5a82"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "19a5c5eca3f9dc48c2d57e50835c3967"
server_key_iv: "575a004bcffa23bfd1e42fac22c4f45e"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.500463 18043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:01:03.501305 18058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.501515 18043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:03.501612 18043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "9b86aeb5642341d79a592e6bf96b5a82"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "19a5c5eca3f9dc48c2d57e50835c3967"
server_key_iv: "575a004bcffa23bfd1e42fac22c4f45e"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.501684 18043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:03.519661 18043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:03.521687 18043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:03.521836 18043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:03.522056 18043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:03.522349 18043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:03.522382 18043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.522413 18043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:03.522435 18043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.527899 18043 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:35871
I20260925 22:01:03.527963 18171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:35871 every 8 connection(s)
I20260925 22:01:03.528254 18043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:01:03.531656 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18043
I20260925 22:01:03.531755 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:01:03.531950 9665 external_mini_cluster.cc:1428] Setting key 338fefc689d3f662e8ff547aa976134d
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:03.538064 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.528959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:33213 (local address 127.9.112.126:45243)
0925 22:01:03.529004 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:03.529006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:03.529206 (+ 200us) server_negotiation.cc:407] Connection header received
0925 22:01:03.529741 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:03.529742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:03.529753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:03.529818 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:03.530852 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.531389 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.532743 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.533004 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.533076 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.535441 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:03.535443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:03.535444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:03.535452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:03.536799 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.537169 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.537171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.537171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.537227 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.537447 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.537448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.537448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.537557 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:03.537610 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:03.537887 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 22:01:03.537971 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:01:03.538239 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.528934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:33213)
0925 22:01:03.529086 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:03.529129 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:03.529597 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:03.529870 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:03.529873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:03.530505 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:03.530705 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:03.530711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.531504 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.531505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:03.532674 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:03.532678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.533079 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.533079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:03.533205 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.533760 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:03.533768 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:03.535341 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:03.536876 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:03.536883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:03.536886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:03.537104 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:03.537283 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:03.537284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:03.537284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:03.537393 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:03.537611 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:03.537618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:03.537770 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 22:01:03.537858 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 22:01:03.538017 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260925 22:01:03.538875 18172 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45243
I20260925 22:01:03.538967 18172 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:03.539139 18172 heartbeater.cc:511] Master 127.9.112.126:45243 requested a full tablet report, sending...
I20260925 22:01:03.539522 17847 ts_manager.cc:194] Registered new tserver with Master: 9b86aeb5642341d79a592e6bf96b5a82 (127.9.112.66:35871)
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
I20260925 22:01:03.539947 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:33213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:01:03.560298 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45243
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:03.642915 18177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:03.643147 18177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:03.643178 18177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:03.644513 18177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:03.644563 18177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:03.644595 18177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:01:03.646129 18177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:03.646392 18177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:03.646629 18177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:03.651793 18183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.651849 18182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.651774 18185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:03.652021 18177 server_base.cc:1034] running on GCE node
I20260925 22:01:03.652241 18177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:03.652506 18177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:03.653657 18177 hybrid_clock.cc:648] HybridClock initialized: now 1790373663653644 us; error 31 us; skew 500 ppm
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:03.656299 18177 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:01:03.656786 18177 webserver.cc:460] Webserver started at http://127.9.112.67:34347/ using document root <none> and password file <none>
I20260925 22:01:03.657011 18177 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:03.657058 18177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:03.657181 18177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:03.658237 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "80ee35de9cb74cab99f4a786b2920d0e"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "d8dcacf86c22bcc64c3ef5a448f8b9f5"
server_key_iv: "5d72c1ab82b8cc437ed301ddc9bfe5ff"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.658563 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "80ee35de9cb74cab99f4a786b2920d0e"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "d8dcacf86c22bcc64c3ef5a448f8b9f5"
server_key_iv: "5d72c1ab82b8cc437ed301ddc9bfe5ff"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.659883 18177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:01:03.660633 18192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.660777 18177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:03.660859 18177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "80ee35de9cb74cab99f4a786b2920d0e"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "d8dcacf86c22bcc64c3ef5a448f8b9f5"
server_key_iv: "5d72c1ab82b8cc437ed301ddc9bfe5ff"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.660920 18177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:03.683863 18177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:03.686049 18177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:03.686216 18177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:03.686484 18177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:03.686806 18177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:03.686842 18177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.686872 18177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:03.686895 18177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.692677 18177 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:35731
I20260925 22:01:03.692723 18305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:35731 every 8 connection(s)
I20260925 22:01:03.693051 18177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:01:03.695518 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18177
I20260925 22:01:03.695618 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:01:03.695801 9665 external_mini_cluster.cc:1428] Setting key f2f686d2460896ec6614df8e62d293df
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:03.702963 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.693796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:34675 (local address 127.9.112.126:45243)
0925 22:01:03.693873 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:03.693876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:03.694057 (+ 181us) server_negotiation.cc:407] Connection header received
0925 22:01:03.694553 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:03.694554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:03.694564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:03.694626 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:03.695672 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.696245 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.697302 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.697508 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.697567 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.699918 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:03.699920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:03.699920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:03.699927 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:03.701430 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.701950 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.701951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.701952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.702023 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:03.702363 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:03.702364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:03.702364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:03.702486 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:03.702537 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:03.702818 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 22:01:03.702895 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:01:03.703140 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:03.693742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:34675)
0925 22:01:03.693916 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:03.693958 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:03.694444 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:03.694705 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:03.694709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:03.695315 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:03.695517 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:03.695522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.696366 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.696367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:03.697212 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:03.697215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:03.697583 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:03.697584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:03.697716 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:03.698261 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:03.698270 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:03.699729 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:03.701531 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:03.701539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:03.701542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:03.701818 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:03.702110 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:03.702111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:03.702112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:03.702281 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:03.702554 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:03.702560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:03.702700 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 22:01:03.702791 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:01:03.702934 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260925 22:01:03.703617 18306 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45243
I20260925 22:01:03.703717 18306 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:03.703879 18306 heartbeater.cc:511] Master 127.9.112.126:45243 requested a full tablet report, sending...
I20260925 22:01:03.704244 17847 ts_manager.cc:194] Registered new tserver with Master: 80ee35de9cb74cab99f4a786b2920d0e (127.9.112.67:35731)
I20260925 22:01:03.704650 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:34675
I20260925 22:01:03.707676 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab.
Entry for principal oryx/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab.
WARNING: no policy specified for oryx/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab.
Entry for principal oryx/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab.
WARNING: no policy specified for oryx/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab.
Entry for principal oryx/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab.
I20260925 22:01:03.784329 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 17909
I20260925 22:01:03.788442 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18043
I20260925 22:01:03.792887 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18177
I20260925 22:01:03.796561 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 17815
I20260925 22:01:03.800832 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45243
--webserver_interface=127.9.112.126
--webserver_port=44545
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:03.882287 18315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:03.882517 18315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:03.882540 18315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:03.883802 18315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:01:03.883854 18315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:03.883867 18315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:03.883878 18315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:01:03.883889 18315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:01:03.885411 18315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:45243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=44545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:03.885620 18315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:03.885813 18315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:01:03.891088 18315 server_base.cc:1034] running on GCE node
W20260925 22:01:03.891111 18321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.891048 18320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:03.891052 18323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:03.891604 18315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:03.891914 18315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:03.893077 18315 hybrid_clock.cc:648] HybridClock initialized: now 1790373663893041 us; error 48 us; skew 500 ppm
Sep 25 22:01:03 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373663, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:03.895838 18315 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:01:03.896313 18315 webserver.cc:460] Webserver started at http://127.9.112.126:44545/ using document root <none> and password file <none>
I20260925 22:01:03.896582 18315 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:03.896648 18315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:03.898111 18315 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:03.899024 18330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:03.899282 18315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:03.899385 18315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "dc08e654617d421abc3b032063195d5b"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "50f35367fefbbab792f34c7b4e7000ec"
server_key_iv: "fad0276c12fb58fac166a5e1339f5e89"
server_key_version: "encryptionkey@0"
I20260925 22:01:03.899757 18315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:03.907569 18315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:03.909598 18315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:03.909746 18315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:03.913501 18315 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:45243
I20260925 22:01:03.913532 18382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:45243 every 8 connection(s)
I20260925 22:01:03.913894 18315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:01:03.915299 18383 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:01:03.915335 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18315
I20260925 22:01:03.915584 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:38181
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:45243
--webserver_port=33479
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:01:03.916010 18383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Bootstrap starting.
I20260925 22:01:03.924015 18383 log.cc:824] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:03.925835 18383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:01:03.926110 18383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Bootstrap complete.
I20260925 22:01:03.928224 18383 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.928514 18383 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc08e654617d421abc3b032063195d5b, State: Initialized, Role: FOLLOWER
I20260925 22:01:03.928678 18383 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.928743 18383 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:01:03.928785 18383 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:01:03.928829 18383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:01:03.929864 18383 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.929955 18383 leader_election.cc:302] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc08e654617d421abc3b032063195d5b; no voters:
I20260925 22:01:03.930119 18383 leader_election.cc:288] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:01:03.930159 18387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:01:03.930274 18387 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [term 2 LEADER]: Becoming Leader. State: Replica: dc08e654617d421abc3b032063195d5b, State: Running, Role: LEADER
I20260925 22:01:03.930358 18387 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } }
I20260925 22:01:03.930392 18383 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:01:03.930620 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc08e654617d421abc3b032063195d5b. Latest consensus state: current_term: 2 leader_uuid: "dc08e654617d421abc3b032063195d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } } }
I20260925 22:01:03.930696 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:03.930682 18389 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc08e654617d421abc3b032063195d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc08e654617d421abc3b032063195d5b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45243 } } }
I20260925 22:01:03.930926 18389 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:03.930874 18391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:01:03.931492 18391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:01:03.931632 18391 catalog_manager.cc:1259] Loaded cluster ID: d45e534392a84b0ab7506e5f1b486078
I20260925 22:01:03.931684 18391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:01:03.932862 18391 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:01:03.933038 18391 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc08e654617d421abc3b032063195d5b: Loaded TSK: 0
I20260925 22:01:03.933239 18391 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:01:03.999378 18385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:03.999605 18385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:03.999639 18385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:04.001020 18385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:04.001078 18385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:04.001118 18385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:01:04.002774 18385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.65.keytab
--principal=oryx/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:38181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=33479
--tserver_master_addrs=127.9.112.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:04.003039 18385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:04.003255 18385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:04.008307 18410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:04.008249 18412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:04.008245 18409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:04.008881 18385 server_base.cc:1034] running on GCE node
I20260925 22:01:04.009094 18385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:04.009379 18385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:04.010568 18385 hybrid_clock.cc:648] HybridClock initialized: now 1790373664010548 us; error 34 us; skew 500 ppm
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373664, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:04.013685 18385 init.cc:377] Logged in from keytab as oryx/127.9.112.65@KRBTEST.COM (short username oryx)
I20260925 22:01:04.014254 18385 webserver.cc:460] Webserver started at http://127.9.112.65:33479/ using document root <none> and password file <none>
I20260925 22:01:04.014518 18385 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:04.014595 18385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:04.016448 18385 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:01:04.017162 18419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.017354 18385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:04.017447 18385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "6dd971fae771464bab48e02d4dd34d36"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "149ff92b6711254051c2719aba0aa55b"
server_key_iv: "d846f033f1852049efe341d3f64f4d7f"
server_key_version: "encryptionkey@0"
I20260925 22:01:04.017783 18385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:04.038365 18385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:04.040623 18385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:04.040777 18385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:04.040995 18385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:04.041298 18385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:04.041332 18385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.041363 18385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:04.041414 18385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.047204 18385 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:38181
I20260925 22:01:04.047251 18532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:38181 every 8 connection(s)
I20260925 22:01:04.047581 18385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:01:04.050701 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18385
I20260925 22:01:04.051002 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:35871
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:45243
--webserver_port=42325
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.058177 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.048328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:37741)
0925 22:01:04.048500 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.048543 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.048990 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.049444 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.049448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.050024 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.050228 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.050233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.052200 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.052201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.053481 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.053485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.054218 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.054219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.054375 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.054997 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.055008 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.057941 (+ 2933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
W20260925 22:01:04.058501 18533 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:45243 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.112.126:45243: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
W20260925 22:01:04.058653 18536 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.048429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:37741 (local address 127.9.112.126:45243)
0925 22:01:04.048642 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.048650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.048685 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:01:04.049198 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.049205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.049260 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.049405 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.050368 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.052051 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.053591 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.054079 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.054192 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.058085 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:37741: BlockingRecv error: recv got EOF from 127.9.112.65:37741 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":92,"threads_started":1}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.064478 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.058756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:43787)
0925 22:01:04.058861 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.058873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.058916 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.059203 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.059204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.059368 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.059429 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.059430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.060457 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.060458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.061205 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.061210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.061708 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.061709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.061772 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.062265 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.062270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.064397 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260925 22:01:04.064587 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.058858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:43787 (local address 127.9.112.126:45243)
0925 22:01:04.059050 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.059052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.059060 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:04.059087 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.059088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.059101 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.059175 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.059520 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.060303 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.061297 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.061620 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.061700 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.064503 (+ 2803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:43787: BlockingRecv error: recv got EOF from 127.9.112.65:43787 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.069514 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.064716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:48123)
0925 22:01:04.064891 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.064903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.064959 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.065085 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.065086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.065243 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.065292 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.065293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.065915 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.065917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.066506 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.066509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.066867 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.066868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.066935 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.067428 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.067432 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.069424 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
W20260925 22:01:04.069588 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.064906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:48123 (local address 127.9.112.126:45243)
0925 22:01:04.064948 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.064950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.064958 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:04.065019 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.065019 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.065029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.065075 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.065363 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.065836 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.066582 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.066774 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.066833 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.069535 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:48123: BlockingRecv error: recv got EOF from 127.9.112.65:48123 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:01:04.069680 18533 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:01:04.136098 18537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:04.136327 18537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:04.136379 18537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:04.137651 18537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:04.137692 18537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:04.137722 18537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:01:04.139263 18537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.66.keytab
--principal=oryx/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:35871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=42325
--tserver_master_addrs=127.9.112.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:04.139498 18537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:04.139711 18537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:04.144795 18545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:04.144840 18544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:04.144945 18537 server_base.cc:1034] running on GCE node
W20260925 22:01:04.144956 18547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:04.145231 18537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:04.145478 18537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:04.146636 18537 hybrid_clock.cc:648] HybridClock initialized: now 1790373664146619 us; error 28 us; skew 500 ppm
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373664, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:04.149281 18537 init.cc:377] Logged in from keytab as oryx/127.9.112.66@KRBTEST.COM (short username oryx)
I20260925 22:01:04.149710 18537 webserver.cc:460] Webserver started at http://127.9.112.66:42325/ using document root <none> and password file <none>
I20260925 22:01:04.149921 18537 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:04.149982 18537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:04.151418 18537 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:04.152092 18554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.152311 18537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:04.152444 18537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "9b86aeb5642341d79a592e6bf96b5a82"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "19a5c5eca3f9dc48c2d57e50835c3967"
server_key_iv: "575a004bcffa23bfd1e42fac22c4f45e"
server_key_version: "encryptionkey@0"
I20260925 22:01:04.152808 18537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:04.164484 18537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:04.166683 18537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:04.166824 18537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:04.167057 18537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:04.167346 18537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:04.167383 18537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.167412 18537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:04.167429 18537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.172950 18537 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:35871
I20260925 22:01:04.173030 18667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:35871 every 8 connection(s)
I20260925 22:01:04.173354 18537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:01:04.178048 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18537
I20260925 22:01:04.178398 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:35731
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:45243
--webserver_port=34347
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:40941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:01:04.181151 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.174061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:34971 (local address 127.9.112.126:45243)
0925 22:01:04.174107 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.174109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.174334 (+ 225us) server_negotiation.cc:407] Connection header received
0925 22:01:04.174829 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.174830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.174846 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.174899 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.175824 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.176313 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.177328 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.177520 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.177576 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.181087 (+ 3511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:34971: BlockingRecv error: recv got EOF from 127.9.112.66:34971 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:01:04.181159 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.174049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:34971)
0925 22:01:04.174188 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.174225 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.174728 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.174976 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.174979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.175539 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.175729 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.175733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.176437 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.176438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.177237 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.177239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.177588 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.177588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.177703 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.178334 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.178343 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.180978 (+ 2635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260925 22:01:04.181392 18668 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:45243 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.112.126:45243: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.186681 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.181610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:47487)
0925 22:01:04.181677 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.181688 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.181732 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.181875 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.181875 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.182019 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.182080 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.182083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.182779 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.182779 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.183573 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.183577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.184049 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.184050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.184136 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.184613 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.184618 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.186603 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 22:01:04.186826 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.181678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:47487 (local address 127.9.112.126:45243)
0925 22:01:04.181721 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.181723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.181734 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:01:04.181794 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.181795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.181808 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.181858 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.182188 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.182695 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.183751 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.183951 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.184014 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.186713 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:47487: BlockingRecv error: recv got EOF from 127.9.112.66:47487 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.191076 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.187024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:36651)
0925 22:01:04.187066 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.187077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.187110 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.187258 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.187259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.187368 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.187409 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.187411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.188011 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.188012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.188533 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.188534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.188840 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.188841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.188890 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.189255 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.189260 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.191001 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 22:01:04.191149 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.187081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:36651 (local address 127.9.112.126:45243)
0925 22:01:04.187134 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.187136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.187144 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:04.187168 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.187169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.187176 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.187220 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.187477 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.187947 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.188599 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.188786 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.188838 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.191087 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:36651: BlockingRecv error: recv got EOF from 127.9.112.66:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260925 22:01:04.191227 18668 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:01:04.262178 18671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:04.262420 18671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:04.262446 18671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:04.263767 18671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:04.263820 18671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:04.263849 18671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:01:04.265467 18671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/krb5kdc/oryx_127.9.112.67.keytab
--principal=oryx/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:35731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=34347
--tserver_master_addrs=127.9.112.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:04.265746 18671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:04.265969 18671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:04.271010 18679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:04.271046 18677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:04.271014 18676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:04.271329 18671 server_base.cc:1034] running on GCE node
I20260925 22:01:04.271520 18671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:04.271761 18671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:04.272925 18671 hybrid_clock.cc:648] HybridClock initialized: now 1790373664272910 us; error 30 us; skew 500 ppm
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373664, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:04.275511 18671 init.cc:377] Logged in from keytab as oryx/127.9.112.67@KRBTEST.COM (short username oryx)
I20260925 22:01:04.276039 18671 webserver.cc:460] Webserver started at http://127.9.112.67:34347/ using document root <none> and password file <none>
I20260925 22:01:04.276260 18671 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:04.276358 18671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:04.277832 18671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:04.278469 18686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.278687 18671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:04.278785 18671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "80ee35de9cb74cab99f4a786b2920d0e"
format_stamp: "Formatted at 2026-09-25 22:01:03 on dist-test-slave-794d"
server_key: "d8dcacf86c22bcc64c3ef5a448f8b9f5"
server_key_iv: "5d72c1ab82b8cc437ed301ddc9bfe5ff"
server_key_version: "encryptionkey@0"
I20260925 22:01:04.279146 18671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:04.302857 18671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:04.305490 18671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:04.305644 18671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:04.305877 18671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:04.306178 18671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:04.306211 18671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.306242 18671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:04.306263 18671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:04.312095 18671 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:35731
I20260925 22:01:04.312155 18799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:35731 every 8 connection(s)
I20260925 22:01:04.312510 18671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:01:04.314134 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18671
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 22:01:04.321272 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.313227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:49999 (local address 127.9.112.126:45243)
0925 22:01:04.313270 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.313272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.313499 (+ 227us) server_negotiation.cc:407] Connection header received
0925 22:01:04.313976 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.313977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.313987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.314027 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.315399 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.316159 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.317603 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.317828 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.317879 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.321177 (+ 3298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:49999: BlockingRecv error: recv got EOF from 127.9.112.67:49999 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:01:04.321507 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.313234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:49999)
0925 22:01:04.313378 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.313418 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.313890 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.314112 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.314116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.314970 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.315273 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.315279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.316271 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.316272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.317518 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.317520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.317919 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.317920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.318030 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.318514 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.318524 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.321066 (+ 2542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
W20260925 22:01:04.321761 18800 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:45243 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.112.126:45243: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
I20260925 22:01:04.322600 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.314655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58698 (local address 127.9.112.126:45243)
0925 22:01:04.314700 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.314701 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.314736 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:01:04.314797 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.314798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.314808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.314852 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.315525 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.316253 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.317389 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.317595 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.317680 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.319026 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:04.319031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:04.319031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:04.319042 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:04.320838 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:04.321246 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:04.321250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:04.321251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:04.321391 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:04.321800 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:04.321801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:04.321802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:04.322078 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:04.322201 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:04.322386 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 22:01:04.322508 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.326900 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.322331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:47175)
0925 22:01:04.322425 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.322435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.322478 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.322589 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.322589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.322757 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.322817 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.322819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.323611 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.323613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.324146 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.324148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.324566 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.324567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.324628 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.325063 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.325067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.326842 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260925 22:01:04.326992 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.322386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:47175 (local address 127.9.112.126:45243)
0925 22:01:04.322431 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.322433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.322465 (+ 32us) server_negotiation.cc:407] Connection header received
0925 22:01:04.322531 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.322531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.322541 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.322587 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.322889 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.323552 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.324232 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.324492 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.324559 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.326931 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:47175: BlockingRecv error: recv got EOF from 127.9.112.67:47175 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:04 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:04.331097 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:04.327165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:37207)
0925 22:01:04.327213 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:04.327224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:04.327265 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:04.327380 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:04.327381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:04.327488 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:04.327520 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.327521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.328124 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.328125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.328697 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:04.328700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.329027 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.329027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:04.329068 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.329423 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:04.329427 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:04.331044 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 22:01:04.331200 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:04.327217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:37207 (local address 127.9.112.126:45243)
0925 22:01:04.327257 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:04.327259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:04.327267 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:04.327318 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:04.327319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:04.327327 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:04.327365 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:04.327569 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.328042 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.328764 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:04.328958 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:04.329011 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:04.331132 (+ 2121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:37207: BlockingRecv error: recv got EOF from 127.9.112.67:37207 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 22:01:04.331239 18800 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 22:01:05 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:05 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:05.074949 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:05.070103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:46777)
0925 22:01:05.070253 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:05.070263 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:05.070317 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:05.070525 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:05.070526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:05.070691 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:05.070739 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:05.070741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.071536 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.071537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:05.072097 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:05.072099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.072519 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.072520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:05.072588 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:05.073107 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:05.073111 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:05.074870 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
W20260925 22:01:05.075047 18805 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:05.070194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:46777 (local address 127.9.112.126:45243)
0925 22:01:05.070308 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:05.070309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:05.070321 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:01:05.070388 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:05.070389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:05.070400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:05.070454 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:05.070878 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.071408 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.072197 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.072438 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.072504 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:05.074974 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:46777: BlockingRecv error: recv got EOF from 127.9.112.65:46777 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
Sep 25 22:01:05 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:05 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:05.197048 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:05.191758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:57487)
0925 22:01:05.191927 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:05.191939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:05.191999 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:05.192246 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:05.192247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:05.192439 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:05.192487 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:05.192489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.193250 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.193251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:05.193849 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:05.193852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.194235 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.194236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:05.194289 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:05.194902 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:05.194908 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:05.196974 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":68,"threads_started":1}
W20260925 22:01:05.197144 18805 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:05.191936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:57487 (local address 127.9.112.126:45243)
0925 22:01:05.192039 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:05.192041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:05.192050 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:05.192077 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:05.192078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:05.192089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:05.192153 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:05.192592 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.193154 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.193942 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.194140 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.194202 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:05.197080 (+ 2878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:57487: BlockingRecv error: recv got EOF from 127.9.112.66:57487 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 25 22:01:05 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:05 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:05.336391 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:05.331613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:36921)
0925 22:01:05.331755 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:05.331766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:05.331834 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:05.331963 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:05.331964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:05.332147 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:05.332196 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:05.332198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.332870 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.332871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:05.333443 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:05.333446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.333915 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.333916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:05.333978 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:05.334459 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:05.334464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:05.336271 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260925 22:01:05.336462 18805 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:05.331670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:36921 (local address 127.9.112.126:45243)
0925 22:01:05.331743 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:05.331746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:05.331848 (+ 102us) server_negotiation.cc:407] Connection header received
0925 22:01:05.331880 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:05.331881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:05.331894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:05.331959 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:05.332259 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.332775 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.333546 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:05.333746 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:05.333897 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:05.336409 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:36921: BlockingRecv error: recv got EOF from 127.9.112.67:36921 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 22:01:06 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:06 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:06.080259 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:06.075546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:37023)
0925 22:01:06.075680 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:06.075691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:06.075760 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:06.075920 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:06.075921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:06.076091 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:06.076140 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:06.076142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.077023 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.077024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:06.077577 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:06.077580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.077922 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.077922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:06.077978 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:06.078445 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:06.078450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:06.080176 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
W20260925 22:01:06.080381 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:06.075641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:37023 (local address 127.9.112.126:45243)
0925 22:01:06.075815 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:06.075817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:06.075823 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:01:06.075841 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:06.075841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:06.075851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:06.075903 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:06.076256 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.076931 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.077670 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.077858 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.077912 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:06.080289 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:37023: BlockingRecv error: recv got EOF from 127.9.112.65:37023 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
Sep 25 22:01:06 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:06 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:06.202179 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:06.197547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:32917)
0925 22:01:06.197667 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:06.197678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:06.197724 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:06.197843 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:06.197844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:06.197994 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:06.198040 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:06.198042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.198712 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.198713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:06.199238 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:06.199241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.199587 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.199588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:06.199635 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:06.200158 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:06.200163 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:06.202117 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
W20260925 22:01:06.202394 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:06.197634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:32917 (local address 127.9.112.126:45243)
0925 22:01:06.197724 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:06.197726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:06.197746 (+ 20us) server_negotiation.cc:407] Connection header received
0925 22:01:06.197769 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:06.197770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:06.197781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:06.197835 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:06.198116 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.198619 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.199326 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.199516 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.199569 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:06.202347 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:32917: BlockingRecv error: recv got EOF from 127.9.112.66:32917 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 25 22:01:06 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:06 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:06.341879 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:06.336896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:49099)
0925 22:01:06.337075 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:06.337089 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:06.337152 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:06.337278 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:06.337279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:06.337450 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:06.337498 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:06.337499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.338162 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.338163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:06.338713 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:06.338716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.339134 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.339135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:06.339230 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:06.339792 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:06.339798 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:06.341819 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
W20260925 22:01:06.342005 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:06.337062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:49099 (local address 127.9.112.126:45243)
0925 22:01:06.337130 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:06.337132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:06.337141 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:06.337200 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:06.337201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:06.337212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:06.337264 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:06.337550 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.338076 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.338808 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:06.339027 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:06.339107 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:06.341931 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:49099: BlockingRecv error: recv got EOF from 127.9.112.67:49099 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 22:01:07 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:07 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:07.085209 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:07.080694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:33137)
0925 22:01:07.080819 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:07.080830 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:07.080897 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:07.081027 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:07.081028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:07.081224 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:07.081274 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:07.081276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.081958 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.081959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:07.082607 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:07.082609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.082933 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.082934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:07.082984 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:07.083459 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:07.083464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:07.085131 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
W20260925 22:01:07.085297 18813 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:07.080746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:33137 (local address 127.9.112.126:45243)
0925 22:01:07.080845 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:07.080848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:07.080878 (+ 30us) server_negotiation.cc:407] Connection header received
0925 22:01:07.080947 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:07.080948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:07.080960 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:07.081020 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:07.081345 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.081881 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.082679 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.082876 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.082930 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:07.085234 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:33137: BlockingRecv error: recv got EOF from 127.9.112.65:33137 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":61,"threads_started":1}
Sep 25 22:01:07 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:07 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:07.208055 18814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:07.202683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:33567)
0925 22:01:07.202809 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:07.202820 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:07.202901 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:07.203050 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:07.203050 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:07.203204 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:07.203255 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:07.203257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.204426 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.204428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:07.205119 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:07.205123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.205611 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.205612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:07.205662 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:07.206144 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:07.206149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:07.207977 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
W20260925 22:01:07.208160 18813 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:07.202786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:33567 (local address 127.9.112.126:45243)
0925 22:01:07.202864 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:07.202866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:07.202884 (+ 18us) server_negotiation.cc:407] Connection header received
0925 22:01:07.202962 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:07.202963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:07.202973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:07.203036 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:07.203385 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.204208 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.205303 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.205520 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.205587 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:07.208105 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:33567: BlockingRecv error: recv got EOF from 127.9.112.66:33567 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 22:01:07 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:07 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:07.346906 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:07.342428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:60379)
0925 22:01:07.342595 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:07.342607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:07.342686 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:07.342826 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:07.342827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:07.342992 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:07.343037 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:07.343039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.343706 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.343707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:07.344257 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:07.344259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.344649 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.344650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:07.344713 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:07.345184 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:07.345188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:07.346842 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":65,"threads_started":1}
W20260925 22:01:07.347043 18813 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:07.342546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:60379 (local address 127.9.112.126:45243)
0925 22:01:07.342603 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:07.342605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:07.342677 (+ 72us) server_negotiation.cc:407] Connection header received
0925 22:01:07.342744 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:07.342745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:07.342759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:07.342807 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:07.343111 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.343629 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.344370 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:07.344566 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:07.344632 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:07.346969 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:60379: BlockingRecv error: recv got EOF from 127.9.112.67:60379 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 25 22:01:08 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:08 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:08.090339 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:08.085733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:52269)
0925 22:01:08.085905 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:08.085916 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:08.085970 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:08.086104 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:08.086104 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:08.086280 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:08.086330 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:08.086332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.086969 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.086969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:08.087503 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:08.087505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.087877 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.087878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:08.087919 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:08.088501 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:08.088506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:08.090259 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
W20260925 22:01:08.090466 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:08.085802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:52269 (local address 127.9.112.126:45243)
0925 22:01:08.085934 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:08.085936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:08.085958 (+ 22us) server_negotiation.cc:407] Connection header received
0925 22:01:08.086032 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:08.086033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:08.086043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:08.086093 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:08.086385 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.086904 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.087605 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.087806 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.087871 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:08.090400 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:52269: BlockingRecv error: recv got EOF from 127.9.112.65:52269 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
Sep 25 22:01:08 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:08 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:08.217519 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:08.208649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:57931)
0925 22:01:08.208832 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:08.208843 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:08.208899 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:08.209049 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:08.209049 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:08.209211 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:08.209276 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:08.209277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.209988 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.209989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:08.210528 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:08.210530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.211007 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.211008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:08.211069 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:08.215569 (+ 4500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:08.215575 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:08.217371 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
W20260925 22:01:08.217558 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:08.208817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:57931 (local address 127.9.112.126:45243)
0925 22:01:08.208900 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:08.208902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:08.208946 (+ 44us) server_negotiation.cc:407] Connection header received
0925 22:01:08.208972 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:08.208973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:08.208984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:08.209027 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:08.209366 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.209908 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.210621 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.210897 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.210970 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:08.217511 (+ 6541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:57931: BlockingRecv error: recv got EOF from 127.9.112.66:57931 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 22:01:08 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:08 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:08.351778 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:08.347425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:58569)
0925 22:01:08.347607 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:08.347620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:08.347698 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:08.347836 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:08.347837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:08.348015 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:08.348060 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:08.348063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.348739 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.348741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:08.349249 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:08.349250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.349557 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.349558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:08.349607 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:08.350041 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:08.350045 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:08.351691 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":64,"threads_started":1}
W20260925 22:01:08.351857 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:08.347530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:58569 (local address 127.9.112.126:45243)
0925 22:01:08.347620 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:08.347623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:08.347690 (+ 67us) server_negotiation.cc:407] Connection header received
0925 22:01:08.347759 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:08.347759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:08.347772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:08.347824 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:08.348132 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.348671 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.349322 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:08.349501 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:08.349553 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:08.351793 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:58569: BlockingRecv error: recv got EOF from 127.9.112.67:58569 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 25 22:01:09 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:09 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:09.096401 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:09.090910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:43155)
0925 22:01:09.091067 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:09.091077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:09.091145 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:09.091434 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:09.091436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:09.091595 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:09.091640 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:09.091641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.092435 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.092436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:09.093053 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:09.093057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.093585 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.093586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:09.093659 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:09.094216 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:09.094221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:09.096274 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
W20260925 22:01:09.096510 18821 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:09.091100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:43155 (local address 127.9.112.126:45243)
0925 22:01:09.091251 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:09.091253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:09.091270 (+ 17us) server_negotiation.cc:407] Connection header received
0925 22:01:09.091293 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:09.091293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:09.091303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:09.091396 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:09.091735 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.092284 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.093180 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.093469 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.093546 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:09.096435 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:43155: BlockingRecv error: recv got EOF from 127.9.112.65:43155 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
Sep 25 22:01:09 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:09 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:09.223816 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:09.218226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:60675)
0925 22:01:09.218420 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:09.218431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:09.218492 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:09.218698 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:09.218699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:09.218925 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:09.218973 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:09.218975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.219919 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.219920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:09.220531 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:09.220533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.220945 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.220947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:09.221011 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:09.221519 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:09.221525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:09.223724 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":66,"threads_started":1}
W20260925 22:01:09.223905 18821 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:09.218438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:60675 (local address 127.9.112.126:45243)
0925 22:01:09.218497 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:09.218499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:09.218528 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:01:09.218554 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:09.218555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:09.218565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:09.218604 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:09.219090 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.219810 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.220625 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.220828 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.220908 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:09.223852 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:60675: BlockingRecv error: recv got EOF from 127.9.112.66:60675 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 22:01:09 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:09 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:09.356974 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:09.352404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:58133)
0925 22:01:09.352535 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:09.352546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:09.352607 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:09.352739 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:09.352740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:09.352902 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:09.352949 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:09.352950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.353609 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.353610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:09.354144 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:09.354146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.354488 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.354489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:09.354534 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:09.354961 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:09.354965 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:09.356913 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
W20260925 22:01:09.357060 18821 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:09.352491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:58133 (local address 127.9.112.126:45243)
0925 22:01:09.352579 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:09.352581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:09.352595 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:01:09.352655 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:09.352656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:09.352668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:09.352717 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:09.353029 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.353538 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.354218 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:09.354416 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:09.354473 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:09.356998 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:58133: BlockingRecv error: recv got EOF from 127.9.112.67:58133 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 22:01:10 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:10 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:10.101579 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:10.096908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:37279)
0925 22:01:10.097034 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:10.097044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:10.097099 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:10.097268 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:10.097269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:10.097433 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:10.097481 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:10.097483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.098223 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.098224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:10.098844 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:10.098847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.099227 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.099229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:10.099284 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:10.099712 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:10.099716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:10.101516 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
W20260925 22:01:10.101686 18825 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:10.096962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:37279 (local address 127.9.112.126:45243)
0925 22:01:10.097061 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:10.097063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:10.097090 (+ 27us) server_negotiation.cc:407] Connection header received
0925 22:01:10.097151 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:10.097152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:10.097189 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:10.097244 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:10.097581 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.098121 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.098941 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.099143 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.099198 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:10.101617 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:37279: BlockingRecv error: recv got EOF from 127.9.112.65:37279 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
Sep 25 22:01:10 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:10 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:10.228938 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:10.224372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:59529)
0925 22:01:10.224525 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:10.224537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:10.224610 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:10.224741 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:10.224742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:10.224913 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:10.224956 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:10.224958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.225574 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.225575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:10.226135 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:10.226138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.226478 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.226479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:10.226533 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:10.226973 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:10.226977 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:10.228863 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
W20260925 22:01:10.229138 18825 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:10.224470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:59529 (local address 127.9.112.126:45243)
0925 22:01:10.224528 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:10.224530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:10.224596 (+ 66us) server_negotiation.cc:407] Connection header received
0925 22:01:10.224672 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:10.224673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:10.224684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:10.224730 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:10.225019 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.225507 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.226236 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.226421 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.226475 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:10.228960 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:59529: BlockingRecv error: recv got EOF from 127.9.112.66:59529 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 22:01:10 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:10 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:10.362640 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:10.357449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:55913)
0925 22:01:10.357622 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:10.357635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:10.357710 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:10.357880 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:10.357881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:10.358037 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:10.358085 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:10.358087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.358969 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.358970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:10.359698 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:10.359706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.360094 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.360095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:10.360153 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:10.360630 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:10.360635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:10.362580 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
W20260925 22:01:10.362845 18825 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:10.357555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:55913 (local address 127.9.112.126:45243)
0925 22:01:10.357651 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:10.357653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:10.357694 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:01:10.357759 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:10.357760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:10.357773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:10.357824 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:10.358180 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.358833 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.359810 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:10.360023 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:10.360088 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:10.362782 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:55913: BlockingRecv error: recv got EOF from 127.9.112.67:55913 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 22:01:11 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:11 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:11.106932 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:11.102118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:40077)
0925 22:01:11.102244 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:11.102257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:11.102324 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:11.102556 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:11.102558 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:11.102724 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:11.102772 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:11.102774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.103510 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.103511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:11.104128 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:11.104130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.104522 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.104523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:11.104574 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:11.105162 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:11.105167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:11.106850 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
W20260925 22:01:11.107012 18829 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:11.102211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40077 (local address 127.9.112.126:45243)
0925 22:01:11.102310 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:11.102312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:11.102321 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:11.102427 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:11.102429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:11.102441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:11.102507 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:11.102885 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.103426 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.104207 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.104454 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.104506 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:11.106952 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:40077: BlockingRecv error: recv got EOF from 127.9.112.65:40077 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":43,"threads_started":1}
Sep 25 22:01:11 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:11 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:11.234395 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:11.229618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:38105)
0925 22:01:11.229750 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:11.229763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:11.229824 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:11.229997 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:11.229998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:11.230189 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:11.230239 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:11.230242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.230951 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.230952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:11.231626 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:11.231635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.232034 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.232035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:11.232093 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:11.232529 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:11.232534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:11.234315 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260925 22:01:11.234483 18829 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:11.229642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:38105 (local address 127.9.112.126:45243)
0925 22:01:11.229789 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:11.229791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:11.229806 (+ 15us) server_negotiation.cc:407] Connection header received
0925 22:01:11.229892 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:11.229893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:11.229907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:11.229988 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:11.230332 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.230847 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.231733 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.231948 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.232028 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:11.234429 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:38105: BlockingRecv error: recv got EOF from 127.9.112.66:38105 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Sep 25 22:01:11 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:11 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:11.367733 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:11.363188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:56987)
0925 22:01:11.363305 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:11.363317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:11.363382 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:11.363506 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:11.363507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:11.363662 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:11.363707 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:11.363709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.364379 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.364380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:11.364897 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:11.364900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.365217 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.365217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:11.365269 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:11.365662 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:11.365665 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:11.367591 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
W20260925 22:01:11.367761 18829 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:11.363292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:56987 (local address 127.9.112.126:45243)
0925 22:01:11.363380 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:11.363382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:11.363390 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:11.363433 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:11.363434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:11.363446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:11.363488 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:11.363781 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.364278 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.364967 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:11.365160 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:11.365213 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:11.367706 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:56987: BlockingRecv error: recv got EOF from 127.9.112.67:56987 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 25 22:01:12 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:12 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:12.112262 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:12.107468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:45775)
0925 22:01:12.107603 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:12.107611 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:12.107652 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:12.107850 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:12.107851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:12.107994 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:12.108041 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:12.108043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.108733 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.108734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:12.109257 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:12.109260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.109712 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.109713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:12.109752 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:12.110172 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:12.110176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:12.112171 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":56,"threads_started":1}
W20260925 22:01:12.112430 18833 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:12.107592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:45775 (local address 127.9.112.126:45243)
0925 22:01:12.107712 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:12.107715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:12.107724 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:12.107751 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:12.107751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:12.107761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:12.107840 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:12.108094 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.108661 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.109347 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.109539 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.109710 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:12.112275 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:45775: BlockingRecv error: recv got EOF from 127.9.112.65:45775 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
Sep 25 22:01:12 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:12 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:12.240003 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:12.234917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:50145)
0925 22:01:12.235056 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:12.235066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:12.235134 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:12.235252 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:12.235252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:12.235410 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:12.235456 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:12.235458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.236264 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.236266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:12.237006 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:12.237009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.237441 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.237442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:12.237493 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:12.238035 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:12.238040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:12.239941 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
W20260925 22:01:12.240144 18833 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:12.235021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:50145 (local address 127.9.112.126:45243)
0925 22:01:12.235065 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:12.235067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:12.235132 (+ 65us) server_negotiation.cc:407] Connection header received
0925 22:01:12.235177 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:12.235178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:12.235191 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:12.235237 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:12.235528 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.236168 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.237138 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.237351 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.237415 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:12.240080 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:50145: BlockingRecv error: recv got EOF from 127.9.112.66:50145 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 22:01:12 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:12 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:12.373118 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:12.368286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:43407)
0925 22:01:12.368490 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:12.368501 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:12.368574 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:12.368793 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:12.368794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:12.368958 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:12.369004 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:12.369006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.369641 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.369642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:12.370148 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:12.370149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.370514 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.370516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:12.370570 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:12.371185 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:12.371190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:12.373044 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":113,"threads_started":1}
W20260925 22:01:12.373255 18833 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:12.368393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:43407 (local address 127.9.112.126:45243)
0925 22:01:12.368484 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:12.368486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:12.368568 (+ 82us) server_negotiation.cc:407] Connection header received
0925 22:01:12.368627 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:12.368628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:12.368642 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:12.368770 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:12.369077 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.369568 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.370269 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:12.370455 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:12.370509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:12.373181 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:43407: BlockingRecv error: recv got EOF from 127.9.112.67:43407 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 22:01:13 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:13 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:13.117506 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:13.112820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:56559)
0925 22:01:13.112947 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:13.112965 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:13.113015 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:13.113168 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:13.113169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:13.113366 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:13.113416 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:13.113419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.114062 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.114063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:13.114638 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:13.114641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.114985 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.114987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:13.115040 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:13.115484 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:13.115490 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:13.117436 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260925 22:01:13.117602 18837 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:13.112908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:56559 (local address 127.9.112.126:45243)
0925 22:01:13.113045 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:13.113047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:13.113055 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:13.113079 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:13.113080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:13.113092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:13.113157 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:13.113477 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.113970 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.114712 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.114900 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.114947 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:13.117534 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:56559: BlockingRecv error: recv got EOF from 127.9.112.65:56559 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
Sep 25 22:01:13 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:13 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:13.249032 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:13.240643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:52709)
0925 22:01:13.240823 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:13.240837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:13.240912 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:13.241145 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:13.241146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:13.241314 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:13.241363 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:13.241365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.242086 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.242087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:13.242621 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:13.242623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.242960 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.242961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:13.243005 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:13.247161 (+ 4156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:13.247167 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:13.248962 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
W20260925 22:01:13.249141 18837 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:13.240848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:52709 (local address 127.9.112.126:45243)
0925 22:01:13.240949 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:13.240951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:13.240965 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:01:13.240995 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:13.240996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:13.241008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:13.241080 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:13.241480 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.241993 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.242693 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.242895 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.242947 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:13.249070 (+ 6123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:52709: BlockingRecv error: recv got EOF from 127.9.112.66:52709 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 25 22:01:13 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:13 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:13.381752 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:13.373691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:33025)
0925 22:01:13.373848 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:13.373862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:13.373929 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:13.374135 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:13.374136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:13.374299 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:13.374351 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:13.374353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.375021 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.375021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:13.375556 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:13.375558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.375911 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.375912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:13.375950 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:13.379723 (+ 3773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:13.379730 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:13.381639 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
W20260925 22:01:13.381811 18837 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:13.373835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:33025 (local address 127.9.112.126:45243)
0925 22:01:13.373907 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:13.373909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:13.373923 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:01:13.374006 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:13.374007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:13.374017 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:13.374085 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:13.374423 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.374942 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.375637 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:13.375823 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:13.375893 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:13.381759 (+ 5866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:33025: BlockingRecv error: recv got EOF from 127.9.112.67:33025 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 22:01:14 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:14 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:14.122704 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:14.118029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:48749)
0925 22:01:14.118172 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:14.118183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:14.118243 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:14.118385 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:14.118386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:14.118550 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:14.118598 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:14.118600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.119272 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.119273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:14.119793 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:14.119795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.120183 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.120184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:14.120231 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:14.120730 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:14.120737 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:14.122623 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
W20260925 22:01:14.122797 18841 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:14.118113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:48749 (local address 127.9.112.126:45243)
0925 22:01:14.118229 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:14.118230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:14.118240 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:01:14.118290 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:14.118291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:14.118301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:14.118361 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:14.118658 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.119196 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.119892 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.120115 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.120167 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:14.122735 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:48749: BlockingRecv error: recv got EOF from 127.9.112.65:48749 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
Sep 25 22:01:14 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:14 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:14.254128 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:14.249498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:44967)
0925 22:01:14.249664 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:14.249677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:14.249749 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:14.249927 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:14.249928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:14.250098 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:14.250142 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:14.250144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.250768 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.250769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:14.251328 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:14.251330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.251659 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.251659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:14.251711 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:14.252241 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:14.252245 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:14.254053 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
W20260925 22:01:14.254215 18841 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:14.249600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:44967 (local address 127.9.112.126:45243)
0925 22:01:14.249668 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:14.249671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:14.249736 (+ 65us) server_negotiation.cc:407] Connection header received
0925 22:01:14.249849 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:14.249850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:14.249860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:14.249905 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:14.250207 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.250694 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.251402 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.251598 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.251654 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:14.254161 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:44967: BlockingRecv error: recv got EOF from 127.9.112.66:44967 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 22:01:14 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:14 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:14.386574 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:14.382216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:34963)
0925 22:01:14.382354 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:14.382367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:14.382432 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:14.382561 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:14.382562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:14.382755 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:14.382802 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:14.382804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.383441 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.383442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:14.383967 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:14.383969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.384289 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.384291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:14.384372 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:14.384823 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:14.384827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:14.386496 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":56,"threads_started":1}
W20260925 22:01:14.386654 18841 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:14.382325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:34963 (local address 127.9.112.126:45243)
0925 22:01:14.382387 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:14.382389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:14.382416 (+ 27us) server_negotiation.cc:407] Connection header received
0925 22:01:14.382491 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:14.382492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:14.382506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:14.382550 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:14.382870 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.383373 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.384046 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:14.384225 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:14.384278 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:14.386598 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:34963: BlockingRecv error: recv got EOF from 127.9.112.67:34963 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 22:01:15 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:15 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:15.128374 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:15.123214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:33021)
0925 22:01:15.123359 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:15.123370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:15.123420 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:15.123577 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:15.123577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:15.123732 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:15.123777 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:15.123778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.124661 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.124663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:15.125200 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:15.125203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.125625 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.125626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:15.125678 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:15.126179 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:15.126184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:15.128257 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
W20260925 22:01:15.128494 18845 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:15.123329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:33021 (local address 127.9.112.126:45243)
0925 22:01:15.123439 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:15.123441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:15.123449 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:15.123474 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:15.123475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:15.123485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:15.123567 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:15.123845 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.124584 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.125306 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.125530 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.125591 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:15.128422 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:33021: BlockingRecv error: recv got EOF from 127.9.112.65:33021 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
Sep 25 22:01:15 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:15 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:15.259523 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:15.254732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:40573)
0925 22:01:15.254876 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:15.254889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:15.254975 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:15.255123 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:15.255124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:15.255353 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:15.255400 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:15.255402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.256056 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.256057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:15.256835 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:15.256838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.257203 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.257204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:15.257259 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:15.257745 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:15.257750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:15.259435 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
W20260925 22:01:15.259593 18845 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:15.254853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:40573 (local address 127.9.112.126:45243)
0925 22:01:15.254909 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:15.254911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:15.254963 (+ 52us) server_negotiation.cc:407] Connection header received
0925 22:01:15.255042 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:15.255043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:15.255055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:15.255108 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:15.255479 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.255977 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.256904 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.257137 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.257192 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:15.259538 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:40573: BlockingRecv error: recv got EOF from 127.9.112.66:40573 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 22:01:15 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:15 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:15.391999 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:15.387185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:42379)
0925 22:01:15.387392 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:15.387409 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:15.387478 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:15.387614 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:15.387614 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:15.387777 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:15.387832 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:15.387834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.388509 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.388510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:15.389137 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:15.389139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.389504 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.389505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:15.389558 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:15.390027 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:15.390031 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:15.391905 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":64,"threads_started":1}
W20260925 22:01:15.392071 18845 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:15.387343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:42379 (local address 127.9.112.126:45243)
0925 22:01:15.387435 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:15.387437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:15.387465 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:01:15.387524 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:15.387525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:15.387537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:15.387591 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:15.387912 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.388438 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.389225 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:15.389432 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:15.389492 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:15.392016 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:42379: BlockingRecv error: recv got EOF from 127.9.112.67:42379 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 25 22:01:16 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:16 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:16.133528 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:16.128944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:38059)
0925 22:01:16.129084 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:16.129095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:16.129144 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:16.129285 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:16.129285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:16.129477 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:16.129525 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:16.129526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.130246 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.130247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:16.130784 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:16.130787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.131193 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.131194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:16.131240 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:16.131720 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:16.131725 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:16.133467 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
W20260925 22:01:16.133651 18849 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:16.129030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:38059 (local address 127.9.112.126:45243)
0925 22:01:16.129139 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:16.129140 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:16.129188 (+ 48us) server_negotiation.cc:407] Connection header received
0925 22:01:16.129211 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:16.129212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:16.129222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:16.129272 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:16.129617 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.130180 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.130869 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.131122 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.131171 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:16.133576 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:38059: BlockingRecv error: recv got EOF from 127.9.112.65:38059 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
Sep 25 22:01:16 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:16 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:16.264658 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:16.260006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:45609)
0925 22:01:16.260123 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:16.260138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:16.260205 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:16.260372 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:16.260373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:16.260545 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:16.260590 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:16.260592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.261285 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.261286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:16.261813 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:16.261815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.262184 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.262185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:16.262247 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:16.262756 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:16.262762 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:16.264579 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
W20260925 22:01:16.264745 18849 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:16.260141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:45609 (local address 127.9.112.126:45243)
0925 22:01:16.260206 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:16.260209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:16.260218 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:16.260290 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:16.260291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:16.260302 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:16.260366 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:16.260663 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.261202 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.261891 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.262086 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.262145 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:16.264692 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:45609: BlockingRecv error: recv got EOF from 127.9.112.66:45609 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 22:01:16 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:16 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:16.397074 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:16.392567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:43857)
0925 22:01:16.392716 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:16.392729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:16.392783 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:16.392934 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:16.392935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:16.393089 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:16.393133 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:16.393134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.393773 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.393774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:16.394313 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:16.394315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.394761 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.394761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:16.394821 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:16.395293 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:16.395298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:16.397013 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":60,"threads_started":1}
W20260925 22:01:16.397213 18849 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:16.392649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:43857 (local address 127.9.112.126:45243)
0925 22:01:16.392702 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:16.392703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:16.392779 (+ 76us) server_negotiation.cc:407] Connection header received
0925 22:01:16.392854 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:16.392855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:16.392866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:16.392918 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:16.393184 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.393679 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.394416 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:16.394606 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:16.394805 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:16.397143 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:43857: BlockingRecv error: recv got EOF from 127.9.112.67:43857 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 25 22:01:17 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:17 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:17.139099 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:17.134032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:40609)
0925 22:01:17.134167 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:17.134177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:17.134225 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:17.134420 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:17.134421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:17.134582 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:17.134628 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:17.134629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.135363 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.135364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:17.135941 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:17.135943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.136311 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.136312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:17.136401 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:17.136981 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:17.136985 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:17.139037 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
W20260925 22:01:17.139261 18853 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:17.134098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40609 (local address 127.9.112.126:45243)
0925 22:01:17.134225 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:17.134227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:17.134269 (+ 42us) server_negotiation.cc:407] Connection header received
0925 22:01:17.134292 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:17.134292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:17.134302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:17.134367 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:17.134765 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.135279 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.136021 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.136231 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.136294 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:17.139183 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:40609: BlockingRecv error: recv got EOF from 127.9.112.65:40609 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
Sep 25 22:01:17 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:17 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:17.270150 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:17.265291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:44605)
0925 22:01:17.265448 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:17.265460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:17.265517 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:17.265810 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:17.265811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:17.265977 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:17.266057 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:17.266059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.266781 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.266782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:17.267309 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:17.267311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.267768 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.267769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:17.267823 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:17.268380 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:17.268386 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:17.270085 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":59,"threads_started":1}
W20260925 22:01:17.270413 18853 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:17.265475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:44605 (local address 127.9.112.126:45243)
0925 22:01:17.265555 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:17.265557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:17.265566 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:17.265595 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:17.265596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:17.265606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:17.265662 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:17.266153 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.266662 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.267444 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.267659 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.267722 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:17.270326 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:44605: BlockingRecv error: recv got EOF from 127.9.112.66:44605 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 22:01:17 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:17 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:17.402472 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:17.397610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:41881)
0925 22:01:17.397816 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:17.397831 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:17.397937 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:17.398126 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:17.398128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:17.398299 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:17.398432 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:17.398436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.399243 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.399244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:17.399869 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:17.399872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.400252 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.400253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:17.400303 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:17.400799 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:17.400803 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:17.402403 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
W20260925 22:01:17.402580 18853 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:17.397782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:41881 (local address 127.9.112.126:45243)
0925 22:01:17.397843 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:17.397879 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:17.397913 (+ 34us) server_negotiation.cc:407] Connection header received
0925 22:01:17.398019 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:17.398020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:17.398034 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:17.398088 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:17.398597 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.399151 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.399966 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:17.400157 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:17.400229 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:17.402501 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:41881: BlockingRecv error: recv got EOF from 127.9.112.67:41881 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 22:01:18 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:18 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:18.144721 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:18.139598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:52083)
0925 22:01:18.139751 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:18.139762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:18.139819 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:18.139989 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:18.139990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:18.140203 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:18.140261 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:18.140263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.140936 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.140936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:18.141491 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:18.141494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.141911 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.141912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:18.141966 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:18.142530 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:18.142535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:18.144649 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
W20260925 22:01:18.144913 18857 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:18.139678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:52083 (local address 127.9.112.126:45243)
0925 22:01:18.139788 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:18.139790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:18.139804 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:01:18.139884 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:18.139885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:18.139898 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:18.139969 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:18.140332 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.140843 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.141587 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.141795 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.141860 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:18.144841 (+ 2981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:52083: BlockingRecv error: recv got EOF from 127.9.112.65:52083 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":48,"threads_started":1}
Sep 25 22:01:18 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:18 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:18.279313 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:18.270584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:42611)
0925 22:01:18.270796 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:18.270808 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:18.270877 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:18.270978 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:18.270979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:18.271135 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:18.271186 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:18.271188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.271921 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.271923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:18.272594 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:18.272597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.272981 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.272982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:18.273037 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:18.277317 (+ 4280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:18.277322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:18.279239 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
W20260925 22:01:18.279420 18857 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:18.270622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:42611 (local address 127.9.112.126:45243)
0925 22:01:18.270665 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:18.270667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:18.270881 (+ 214us) server_negotiation.cc:407] Connection header received
0925 22:01:18.270916 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:18.270916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:18.270926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:18.270970 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:18.271276 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.271783 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.272664 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.272894 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.272944 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:18.279356 (+ 6412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:42611: BlockingRecv error: recv got EOF from 127.9.112.66:42611 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 22:01:18 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:18 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.67@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:18.407672 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:18.402965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.67:57481)
0925 22:01:18.403090 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:18.403102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:18.403172 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:18.403300 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:18.403301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:18.403591 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:18.403661 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:18.403664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.404459 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.404460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:18.404989 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:18.404991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.405378 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.405379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:18.405428 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:18.405923 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:18.405928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:18.407608 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
W20260925 22:01:18.407812 18857 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:18.403007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:57481 (local address 127.9.112.126:45243)
0925 22:01:18.403056 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:18.403058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:18.403167 (+ 109us) server_negotiation.cc:407] Connection header received
0925 22:01:18.403224 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:18.403225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:18.403237 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:18.403291 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:18.403792 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.404294 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.405082 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:18.405276 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:18.405337 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:18.407738 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.67:57481: BlockingRecv error: recv got EOF from 127.9.112.67:57481 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 22:01:19 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:19 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.65@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:19.150103 18860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.145389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.65:45633)
0925 22:01:19.145526 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:19.145536 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:19.145584 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:19.145772 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:19.145787 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:19.145937 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:19.145983 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.145985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.146704 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.146705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.147260 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.147262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.147613 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.147614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.147655 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.148118 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:19.148123 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:19.150042 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":55,"threads_started":1}
W20260925 22:01:19.150208 18861 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:19.145515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:45633 (local address 127.9.112.126:45243)
0925 22:01:19.145625 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.145627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.145635 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:01:19.145658 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.145659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.145668 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.145736 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.146074 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.146607 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.147353 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.147554 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.147608 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.150138 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.65:45633: BlockingRecv error: recv got EOF from 127.9.112.65:45633 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":49,"threads_started":1}
Sep 25 22:01:19 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 22:01:19 dist-test-slave-794d krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.112.66@KRBTEST.COM for oryx/127.9.112.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 22:01:19.284941 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.279989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45243 (local address 127.9.112.66:53689)
0925 22:01:19.280126 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:19.280138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:19.280215 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:19.280426 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:19.280427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:19.280653 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:19.280713 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.280716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.281448 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.281448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.282104 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.282106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.282479 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.282480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.282532 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.283035 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:19.283040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:19.284879 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.112.126:45243: Server oryx/127.9.112.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
W20260925 22:01:19.285090 18861 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:19.280102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:53689 (local address 127.9.112.126:45243)
0925 22:01:19.280212 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.280214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.280256 (+ 42us) server_negotiation.cc:407] Connection header received
0925 22:01:19.280290 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.280291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.280305 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.280415 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.280796 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.281360 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.282193 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.282419 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.282479 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.285022 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.112.66:53689: BlockingRecv error: recv got EOF from 127.9.112.66:53689 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 22:01:19.323927 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18385
I20260925 22:01:19.327757 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18537
I20260925 22:01:19.331590 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18671
I20260925 22:01:19.335175 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18315
2026-09-25T22:01:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17395 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T22:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:01:19Z Disabled control of system clock
I20260925 22:01:19.361007 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38741
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:38741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 22:01:19.442333 18866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:19.442557 18866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:19.442576 18866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:19.443779 18866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:01:19.443823 18866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:19.443835 18866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:19.443847 18866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:01:19.443857 18866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:01:19.445299 18866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:38741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:19.445521 18866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:19.445788 18866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:19.451380 18874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:19.451349 18872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:19.451427 18866 server_base.cc:1034] running on GCE node
W20260925 22:01:19.451380 18871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:19.451973 18866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:19.452281 18866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:19.453437 18866 hybrid_clock.cc:648] HybridClock initialized: now 1790373679453419 us; error 30 us; skew 500 ppm
I20260925 22:01:19.454762 18866 webserver.cc:460] Webserver started at http://127.9.112.126:45301/ using document root <none> and password file <none>
I20260925 22:01:19.455041 18866 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:19.455089 18866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:19.455201 18866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:19.456272 18866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "e6fecc2db7c54cf9bc0ede37ca9da026"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "1a29ac8578ecb73ebb95c789f7c66553"
server_key_iv: "181d484e2ce1d4ef638064cc2ce3793c"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.456652 18866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "e6fecc2db7c54cf9bc0ede37ca9da026"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "1a29ac8578ecb73ebb95c789f7c66553"
server_key_iv: "181d484e2ce1d4ef638064cc2ce3793c"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.457958 18866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:19.458767 18880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.458935 18866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:19.459019 18866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "e6fecc2db7c54cf9bc0ede37ca9da026"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "1a29ac8578ecb73ebb95c789f7c66553"
server_key_iv: "181d484e2ce1d4ef638064cc2ce3793c"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.459085 18866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:19.479313 18866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:19.479628 18866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:19.479766 18866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:19.483435 18866 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:38741
I20260925 22:01:19.483493 18932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:38741 every 8 connection(s)
I20260925 22:01:19.483807 18866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:01:19.484530 18933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:19.485219 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18866
I20260925 22:01:19.485289 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:01:19.485488 9665 external_mini_cluster.cc:1428] Setting key 300386af52c69d1491bfeda3ddec4f79
I20260925 22:01:19.487675 18933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026: Bootstrap starting.
I20260925 22:01:19.488431 18933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:19.488718 18933 log.cc:824] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:19.489395 18933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026: No bootstrap required, opened a new log
I20260925 22:01:19.491495 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.486078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.9.112.126:38741)
0925 22:01:19.486276 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.486285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.486318 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:01:19.486864 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.486873 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.486964 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.487111 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.487281 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.488630 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.489445 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.490041 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.490166 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.490770 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:19.490774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:19.490774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:19.490992 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:19.491036 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:19.491050 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:19.491114 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:19.491181 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 22:01:19.491267 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260925 22:01:19.491595 18933 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38741 } }
I20260925 22:01:19.491734 18933 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:19.491763 18933 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fecc2db7c54cf9bc0ede37ca9da026, State: Initialized, Role: FOLLOWER
I20260925 22:01:19.491884 18933 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38741 } }
I20260925 22:01:19.491959 18933 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:01:19.491986 18933 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:01:19.492027 18933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:19.492654 18933 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38741 } }
I20260925 22:01:19.492744 18933 leader_election.cc:302] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6fecc2db7c54cf9bc0ede37ca9da026; no voters:
I20260925 22:01:19.492871 18933 leader_election.cc:288] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:01:19.492920 18938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:19.493074 18938 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [term 1 LEADER]: Becoming Leader. State: Replica: e6fecc2db7c54cf9bc0ede37ca9da026, State: Running, Role: LEADER
I20260925 22:01:19.493109 18933 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:01:19.493155 18938 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38741 } }
I20260925 22:01:19.493522 18940 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6fecc2db7c54cf9bc0ede37ca9da026. Latest consensus state: current_term: 1 leader_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38741 } } }
I20260925 22:01:19.493523 18939 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fecc2db7c54cf9bc0ede37ca9da026" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38741 } } }
I20260925 22:01:19.493649 18940 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:19.493680 18939 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026 [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:19.493841 18947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:01:19.494292 18947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:01:19.495798 18947 catalog_manager.cc:1347] Generated new cluster ID: 06b50ae2fde74b0caa8d27eb39d3f766
I20260925 22:01:19.495841 18947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:01:19.512111 18947 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:01:19.512687 18947 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:01:19.522356 18947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6fecc2db7c54cf9bc0ede37ca9da026: Generated new TSK 0
I20260925 22:01:19.522521 18947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:01:19.526280 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38741
--builtin_ntp_servers=127.9.112.84:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 22:01:19.620877 18957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:19.621096 18957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:19.621124 18957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:19.622359 18957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:19.622406 18957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:19.622431 18957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:01:19.623775 18957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:19.624027 18957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:19.624325 18957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:19.629122 18962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:19.629259 18963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:19.629501 18957 server_base.cc:1034] running on GCE node
W20260925 22:01:19.629122 18965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:19.629768 18957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:19.629999 18957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:19.631150 18957 hybrid_clock.cc:648] HybridClock initialized: now 1790373679631135 us; error 32 us; skew 500 ppm
I20260925 22:01:19.632550 18957 webserver.cc:460] Webserver started at http://127.9.112.65:35901/ using document root <none> and password file <none>
I20260925 22:01:19.632791 18957 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:19.632843 18957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:19.633157 18957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:19.634269 18957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "e2a35ccff6b6448c9eddc5f58feac2d4"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "f7ff969072f9efadbb39dd3996409751"
server_key_iv: "23894a3f98db46f32f3894d3900e60d7"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.634630 18957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "e2a35ccff6b6448c9eddc5f58feac2d4"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "f7ff969072f9efadbb39dd3996409751"
server_key_iv: "23894a3f98db46f32f3894d3900e60d7"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.635954 18957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:01:19.636739 18971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.636938 18957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:01:19.637025 18957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "e2a35ccff6b6448c9eddc5f58feac2d4"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "f7ff969072f9efadbb39dd3996409751"
server_key_iv: "23894a3f98db46f32f3894d3900e60d7"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.637096 18957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:19.647814 18957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:19.648142 18957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:19.648265 18957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:19.648519 18957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:19.648850 18957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:19.648893 18957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.648923 18957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:19.648947 18957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.654955 18957 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:37775
I20260925 22:01:19.654994 19084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:37775 every 8 connection(s)
I20260925 22:01:19.655362 18957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:01:19.660527 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.656104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:59675 (local address 127.9.112.126:38741)
0925 22:01:19.656157 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.656177 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.656417 (+ 240us) server_negotiation.cc:407] Connection header received
0925 22:01:19.656946 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.656947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.656958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.657021 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.657649 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.658211 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.659246 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.659453 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.659514 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.660240 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:19.660242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:19.660242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:19.660246 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:19.660262 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:19.660267 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:19.660317 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:19.660433 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 22:01:19.660463 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:01:19.660717 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.656083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38741 (local address 127.9.112.65:59675)
0925 22:01:19.656279 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:19.656319 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:19.656820 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:19.657084 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:19.657087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:19.657095 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:19.657562 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.657566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.658313 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.658314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.659160 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.659163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.659516 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.659517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.659623 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.660099 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:01:19.660099 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:19.660128 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:01:19.660150 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:19.660323 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:19.660326 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:19.660329 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:01:19.660430 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:01:19.660519 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260925 22:01:19.661171 19085 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38741
I20260925 22:01:19.661288 19085 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:19.661487 19085 heartbeater.cc:511] Master 127.9.112.126:38741 requested a full tablet report, sending...
I20260925 22:01:19.661939 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 18957
I20260925 22:01:19.662034 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:01:19.662165 18897 ts_manager.cc:194] Registered new tserver with Master: e2a35ccff6b6448c9eddc5f58feac2d4 (127.9.112.65:37775)
I20260925 22:01:19.662242 9665 external_mini_cluster.cc:1428] Setting key ddd5bcba58d3c5879113f713bc6abd7b
I20260925 22:01:19.662885 18897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.65:59675
I20260925 22:01:19.663340 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38741
--builtin_ntp_servers=127.9.112.84:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 22:01:19.744421 19088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:19.744642 19088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:19.744673 19088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:19.745855 19088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:19.745908 19088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:19.745942 19088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:01:19.747303 19088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:19.747558 19088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:19.747850 19088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:19.752771 19094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:19.752866 19096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:19.752794 19093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:19.753062 19088 server_base.cc:1034] running on GCE node
I20260925 22:01:19.753237 19088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:19.753461 19088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:19.754606 19088 hybrid_clock.cc:648] HybridClock initialized: now 1790373679754588 us; error 26 us; skew 500 ppm
I20260925 22:01:19.755856 19088 webserver.cc:460] Webserver started at http://127.9.112.66:40439/ using document root <none> and password file <none>
I20260925 22:01:19.756083 19088 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:19.756132 19088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:19.756253 19088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:19.757427 19088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "cd6c4b18d4e241f097ed2f173301fb7e"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "8d2b2e74e9e0453d1228d9884cffabc5"
server_key_iv: "73357c979e9f1a5ca2da6c5bd4cd8fc9"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.757756 19088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "cd6c4b18d4e241f097ed2f173301fb7e"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "8d2b2e74e9e0453d1228d9884cffabc5"
server_key_iv: "73357c979e9f1a5ca2da6c5bd4cd8fc9"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.759001 19088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:01:19.759800 19102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.759972 19088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:19.760053 19088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "cd6c4b18d4e241f097ed2f173301fb7e"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "8d2b2e74e9e0453d1228d9884cffabc5"
server_key_iv: "73357c979e9f1a5ca2da6c5bd4cd8fc9"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.760115 19088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:19.778650 19088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:19.778954 19088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:19.779084 19088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:19.779313 19088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:19.779627 19088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:19.779659 19088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.779688 19088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:19.779706 19088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.785148 19088 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:41317
I20260925 22:01:19.785199 19215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:41317 every 8 connection(s)
I20260925 22:01:19.785539 19088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:01:19.787932 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19088
I20260925 22:01:19.788024 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:01:19.788203 9665 external_mini_cluster.cc:1428] Setting key a701045ec3ca6f173802f3a266d581ef
I20260925 22:01:19.789120 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38741
--builtin_ntp_servers=127.9.112.84:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 22:01:19.791386 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.786388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:55803 (local address 127.9.112.126:38741)
0925 22:01:19.786446 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.786447 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.786683 (+ 236us) server_negotiation.cc:407] Connection header received
0925 22:01:19.787187 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.787188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.787199 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.787249 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.787975 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.788648 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.789953 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.790168 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.790251 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.791099 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:19.791101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:19.791101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:19.791106 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:19.791123 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:19.791127 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:19.791170 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:19.791276 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:01:19.791305 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:01:19.791572 19218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.786367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38741 (local address 127.9.112.66:55803)
0925 22:01:19.786573 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:19.786607 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:19.787091 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:19.787354 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:19.787357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:19.787365 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:19.787867 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.787874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.788728 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.788729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.789867 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.789870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.790271 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.790272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.790404 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.790958 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:01:19.790959 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:19.790988 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:01:19.791012 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:19.791185 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:19.791190 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:19.791193 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:01:19.791250 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:01:19.791360 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260925 22:01:19.791990 19216 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38741
I20260925 22:01:19.792073 19216 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:19.792245 19216 heartbeater.cc:511] Master 127.9.112.126:38741 requested a full tablet report, sending...
I20260925 22:01:19.792627 18897 ts_manager.cc:194] Registered new tserver with Master: cd6c4b18d4e241f097ed2f173301fb7e (127.9.112.66:41317)
I20260925 22:01:19.793110 18897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.66:55803
W20260925 22:01:19.869151 19219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:19.869383 19219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:19.869405 19219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:19.870563 19219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:19.870599 19219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:19.870621 19219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:01:19.871896 19219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:19.872087 19219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:19.872368 19219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:19.877312 19224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:19.877235 19227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:19.877238 19225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:19.877516 19219 server_base.cc:1034] running on GCE node
I20260925 22:01:19.877713 19219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:19.877940 19219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:19.879088 19219 hybrid_clock.cc:648] HybridClock initialized: now 1790373679879075 us; error 28 us; skew 500 ppm
I20260925 22:01:19.880318 19219 webserver.cc:460] Webserver started at http://127.9.112.67:33777/ using document root <none> and password file <none>
I20260925 22:01:19.880558 19219 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:19.880609 19219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:19.880729 19219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:19.881829 19219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "23e8e6e43459452bb55c116feea10979"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "1deb9b3bb4599e4a4b3773b4f5493637"
server_key_iv: "a91db1727689ed4cdc0757f9804684c3"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.882160 19219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "23e8e6e43459452bb55c116feea10979"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "1deb9b3bb4599e4a4b3773b4f5493637"
server_key_iv: "a91db1727689ed4cdc0757f9804684c3"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.883523 19219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:01:19.884441 19233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.884600 19219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:19.884752 19219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "23e8e6e43459452bb55c116feea10979"
format_stamp: "Formatted at 2026-09-25 22:01:19 on dist-test-slave-794d"
server_key: "1deb9b3bb4599e4a4b3773b4f5493637"
server_key_iv: "a91db1727689ed4cdc0757f9804684c3"
server_key_version: "encryptionkey@0"
I20260925 22:01:19.884822 19219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:19.895390 19219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:19.895699 19219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:19.895821 19219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:19.896029 19219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:19.896332 19219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:19.896448 19219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.896479 19219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:19.896499 19219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:19.901968 19219 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:45623
I20260925 22:01:19.902046 19346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:45623 every 8 connection(s)
I20260925 22:01:19.902390 19219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:01:19.904035 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19219
I20260925 22:01:19.904127 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:01:19.904322 9665 external_mini_cluster.cc:1428] Setting key 37c1b1119e73b460611d599edf631c1d
I20260925 22:01:19.907924 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.903093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:53931 (local address 127.9.112.126:38741)
0925 22:01:19.903139 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.903141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.903395 (+ 254us) server_negotiation.cc:407] Connection header received
0925 22:01:19.903886 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.903887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.903897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.903953 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.904834 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.905519 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.906550 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.906789 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.906861 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.907665 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:19.907666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:19.907667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:19.907672 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:19.907687 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:19.907692 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:19.907746 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:19.907831 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 22:01:19.907854 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:01:19.908102 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:19.903065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38741 (local address 127.9.112.67:53931)
0925 22:01:19.903245 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:19.903278 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:19.903770 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:19.904031 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:19.904036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:19.904046 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:19.904760 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.904767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.905615 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.905616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.906443 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:19.906446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:19.906865 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:19.906866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:19.906975 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:19.907497 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:01:19.907497 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:19.907528 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:01:19.907552 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:19.907757 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:19.907761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:19.907763 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:01:19.907812 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:01:19.907907 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260925 22:01:19.910583 19347 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38741
I20260925 22:01:19.910742 19347 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:19.910916 19347 heartbeater.cc:511] Master 127.9.112.126:38741 requested a full tablet report, sending...
I20260925 22:01:19.911319 18897 ts_manager.cc:194] Registered new tserver with Master: 23e8e6e43459452bb55c116feea10979 (127.9.112.67:45623)
I20260925 22:01:19.911696 18897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.67:53931
I20260925 22:01:19.916203 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 22:01:19.918078 18936 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:19.917613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.9.112.126:38741)
0925 22:01:19.917659 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:19.917661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:19.917804 (+ 143us) server_negotiation.cc:407] Connection header received
0925 22:01:19.917825 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:19.917826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:19.917834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:19.917887 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:19.918004 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43702: BlockingRecv error: recv got EOF from 127.0.0.1:43702 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:01:19.918159 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18957
I20260925 22:01:19.922485 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19088
I20260925 22:01:19.926489 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19219
I20260925 22:01:19.930171 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 18866
2026-09-25T22:01:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (593 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T22:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:01:19Z Disabled control of system clock
I20260925 22:01:19.955075 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:41965
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:41965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:01:20.038254 19358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:20.038496 19358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:20.038527 19358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:20.039719 19358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:01:20.039762 19358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:20.039778 19358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:20.039794 19358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:01:20.039809 19358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:01:20.041198 19358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:41965
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:41965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:20.041465 19358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:20.041754 19358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:20.046605 19364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.046674 19363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.046623 19366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:20.046654 19358 server_base.cc:1034] running on GCE node
I20260925 22:01:20.047058 19358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:20.047333 19358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:20.048468 19358 hybrid_clock.cc:648] HybridClock initialized: now 1790373680048448 us; error 36 us; skew 500 ppm
I20260925 22:01:20.049691 19358 webserver.cc:460] Webserver started at http://127.9.112.126:36367/ using document root <none> and password file <none>
I20260925 22:01:20.049965 19358 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:20.050015 19358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:20.050145 19358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:20.051193 19358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "f7808ecc557247eca997f576ae02886b"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "a44db901dcca0f29c0ae2d8311a8f366"
server_key_iv: "50e1c8ed717f1dd59a6fb2f2f08ec77e"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.051527 19358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "f7808ecc557247eca997f576ae02886b"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "a44db901dcca0f29c0ae2d8311a8f366"
server_key_iv: "50e1c8ed717f1dd59a6fb2f2f08ec77e"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.052829 19358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:01:20.053607 19372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.053797 19358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:01:20.053900 19358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "f7808ecc557247eca997f576ae02886b"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "a44db901dcca0f29c0ae2d8311a8f366"
server_key_iv: "50e1c8ed717f1dd59a6fb2f2f08ec77e"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.053961 19358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:20.057202 19358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:20.057449 19358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:20.057569 19358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:20.061268 19358 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:41965
I20260925 22:01:20.061300 19424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:41965 every 8 connection(s)
I20260925 22:01:20.061621 19358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:01:20.062247 19425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:20.065037 19425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b: Bootstrap starting.
I20260925 22:01:20.065585 19425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:20.065860 19425 log.cc:824] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:20.066506 19425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b: No bootstrap required, opened a new log
I20260925 22:01:20.068105 19425 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7808ecc557247eca997f576ae02886b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41965 } }
I20260925 22:01:20.068251 19425 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:20.068274 19425 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7808ecc557247eca997f576ae02886b, State: Initialized, Role: FOLLOWER
I20260925 22:01:20.068459 19425 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7808ecc557247eca997f576ae02886b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41965 } }
I20260925 22:01:20.068532 19425 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:01:20.068563 19425 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:01:20.068609 19425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:20.069170 19425 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7808ecc557247eca997f576ae02886b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41965 } }
I20260925 22:01:20.069254 19425 leader_election.cc:302] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7808ecc557247eca997f576ae02886b; no voters:
I20260925 22:01:20.069410 19425 leader_election.cc:288] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:01:20.069453 19428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:20.069453 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19358
I20260925 22:01:20.069528 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:01:20.069617 19428 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [term 1 LEADER]: Becoming Leader. State: Replica: f7808ecc557247eca997f576ae02886b, State: Running, Role: LEADER
I20260925 22:01:20.069679 19425 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:01:20.069707 9665 external_mini_cluster.cc:1428] Setting key 8e67932bf6e02503ea8407a93b82d94c
I20260925 22:01:20.069691 19428 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7808ecc557247eca997f576ae02886b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41965 } }
I20260925 22:01:20.069988 19428 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7808ecc557247eca997f576ae02886b. Latest consensus state: current_term: 1 leader_uuid: "f7808ecc557247eca997f576ae02886b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7808ecc557247eca997f576ae02886b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41965 } } }
I20260925 22:01:20.070134 19428 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:20.070423 19438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:01:20.070741 19430 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7808ecc557247eca997f576ae02886b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7808ecc557247eca997f576ae02886b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41965 } } }
I20260925 22:01:20.070861 19430 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:20.071039 19438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:01:20.072597 19438 catalog_manager.cc:1347] Generated new cluster ID: 35a6c9ea85584aca887f51d0a51c40a3
I20260925 22:01:20.072651 19438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:01:20.072638 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.070358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.9.112.126:41965)
0925 22:01:20.070587 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:20.070596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:20.070841 (+ 245us) server_negotiation.cc:407] Connection header received
0925 22:01:20.070962 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:20.070969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:20.071016 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:20.071144 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:20.072157 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:20.072161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:20.072161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:20.072165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:20.072199 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:20.072209 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:20.072257 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:20.072391 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 22:01:20.072451 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":77,"threads_started":1}
I20260925 22:01:20.081210 19438 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:01:20.081429 19438 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:01:20.090924 19438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7808ecc557247eca997f576ae02886b: Generated new TSK 0
I20260925 22:01:20.091120 19438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:01:20.099088 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41965
--builtin_ntp_servers=127.9.112.84:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:01:20.178622 19449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:20.178845 19449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:20.178876 19449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:20.180096 19449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:20.180141 19449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:20.180176 19449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:01:20.181613 19449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:20.181864 19449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:20.182137 19449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:20.187254 19457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.187242 19455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.187244 19454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:20.187500 19449 server_base.cc:1034] running on GCE node
I20260925 22:01:20.187717 19449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:20.187958 19449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:20.189095 19449 hybrid_clock.cc:648] HybridClock initialized: now 1790373680189080 us; error 30 us; skew 500 ppm
I20260925 22:01:20.190299 19449 webserver.cc:460] Webserver started at http://127.9.112.65:40893/ using document root <none> and password file <none>
I20260925 22:01:20.190515 19449 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:20.190563 19449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:20.190688 19449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:20.191773 19449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "2bb19a3f49c54ab5b6a3a7b2cbc41695"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "c31d3580d6e4edad77a18c2339bcd417"
server_key_iv: "3647b54bcb112c73ecebfe1f75050140"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.192255 19449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "2bb19a3f49c54ab5b6a3a7b2cbc41695"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "c31d3580d6e4edad77a18c2339bcd417"
server_key_iv: "3647b54bcb112c73ecebfe1f75050140"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.193537 19449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:01:20.194235 19463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.194393 19449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 22:01:20.194489 19449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "2bb19a3f49c54ab5b6a3a7b2cbc41695"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "c31d3580d6e4edad77a18c2339bcd417"
server_key_iv: "3647b54bcb112c73ecebfe1f75050140"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.194558 19449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:20.198895 19449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:20.199157 19449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:20.199280 19449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:20.199473 19449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:20.199822 19449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:20.199862 19449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.199896 19449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:20.199919 19449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.206012 19449 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:35005
I20260925 22:01:20.206071 19576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:35005 every 8 connection(s)
I20260925 22:01:20.206370 19449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:01:20.209467 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.207041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:50415 (local address 127.9.112.126:41965)
0925 22:01:20.207119 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:20.207121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:20.207448 (+ 327us) server_negotiation.cc:407] Connection header received
0925 22:01:20.207868 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:20.207869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:20.207877 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:20.207942 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:20.209228 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:20.209229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:20.209230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:20.209234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:20.209253 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:20.209258 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:20.209310 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:20.209378 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 22:01:20.209394 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 22:01:20.209656 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.207027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:41965 (local address 127.9.112.65:50415)
0925 22:01:20.207248 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:20.207283 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:20.207751 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:20.208006 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:20.208009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:20.208017 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:20.209072 (+ 1055us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:01:20.209073 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:20.209102 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:01:20.209133 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:20.209312 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:20.209318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:20.209321 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:01:20.209390 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:01:20.209452 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260925 22:01:20.210111 19577 heartbeater.cc:347] Connected to a master server at 127.9.112.126:41965
I20260925 22:01:20.210199 19577 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:20.210311 19577 heartbeater.cc:511] Master 127.9.112.126:41965 requested a full tablet report, sending...
I20260925 22:01:20.210767 19389 ts_manager.cc:194] Registered new tserver with Master: 2bb19a3f49c54ab5b6a3a7b2cbc41695 (127.9.112.65:35005)
I20260925 22:01:20.213730 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19449
I20260925 22:01:20.213815 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:01:20.213961 9665 external_mini_cluster.cc:1428] Setting key e9371faafccec7875d8ba6091396fe3d
I20260925 22:01:20.214893 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41965
--builtin_ntp_servers=127.9.112.84:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:01:20.294392 19580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:20.294611 19580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:20.294633 19580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:20.295794 19580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:20.295825 19580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:20.295850 19580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:01:20.297200 19580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:20.297392 19580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:20.297639 19580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:20.303081 19585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.303077 19588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.303071 19586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:20.303107 19580 server_base.cc:1034] running on GCE node
I20260925 22:01:20.303519 19580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:20.303759 19580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:20.304924 19580 hybrid_clock.cc:648] HybridClock initialized: now 1790373680304908 us; error 27 us; skew 500 ppm
I20260925 22:01:20.306263 19580 webserver.cc:460] Webserver started at http://127.9.112.66:45573/ using document root <none> and password file <none>
I20260925 22:01:20.306627 19580 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:20.306695 19580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:20.306829 19580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:20.307983 19580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "a9c55bb38f394456bedfcbe74f90a46a"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "0f97666d0916b38b08939e62ad33d439"
server_key_iv: "199dd9205dfd778f39367ef291c0897e"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.308310 19580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "a9c55bb38f394456bedfcbe74f90a46a"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "0f97666d0916b38b08939e62ad33d439"
server_key_iv: "199dd9205dfd778f39367ef291c0897e"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.309698 19580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:20.310603 19594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.310774 19580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:20.310850 19580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "a9c55bb38f394456bedfcbe74f90a46a"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "0f97666d0916b38b08939e62ad33d439"
server_key_iv: "199dd9205dfd778f39367ef291c0897e"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.310963 19580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:20.314534 19580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:20.314931 19580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:20.315083 19580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:20.315295 19580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:20.315623 19580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:20.315658 19580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.315685 19580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:20.315719 19580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.321336 19580 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:34661
I20260925 22:01:20.321391 19707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:34661 every 8 connection(s)
I20260925 22:01:20.321728 19580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:01:20.324576 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.322413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:46885 (local address 127.9.112.126:41965)
0925 22:01:20.322477 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:20.322480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:20.322712 (+ 232us) server_negotiation.cc:407] Connection header received
0925 22:01:20.323183 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:20.323185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:20.323190 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:20.323247 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:20.324300 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:20.324301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:20.324302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:20.324305 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:20.324321 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:20.324326 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:20.324446 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:20.324506 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 22:01:20.324519 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:01:20.324747 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.322407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:41965 (local address 127.9.112.66:46885)
0925 22:01:20.322597 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:20.322631 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:20.323080 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:20.323315 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:20.323318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:20.323327 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:20.324157 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:01:20.324157 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:20.324186 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:01:20.324215 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:20.324444 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:20.324448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:20.324450 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:01:20.324506 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:01:20.324559 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260925 22:01:20.325166 19708 heartbeater.cc:347] Connected to a master server at 127.9.112.126:41965
I20260925 22:01:20.325245 19708 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:20.325369 19708 heartbeater.cc:511] Master 127.9.112.126:41965 requested a full tablet report, sending...
I20260925 22:01:20.325771 19389 ts_manager.cc:194] Registered new tserver with Master: a9c55bb38f394456bedfcbe74f90a46a (127.9.112.66:34661)
I20260925 22:01:20.329607 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19580
I20260925 22:01:20.329691 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:01:20.329878 9665 external_mini_cluster.cc:1428] Setting key 25bd4c47233c99a122b9b4488719fe13
I20260925 22:01:20.330888 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41965
--builtin_ntp_servers=127.9.112.84:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 22:01:20.412173 19711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:20.412477 19711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:20.412513 19711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:20.413903 19711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:20.413949 19711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:20.413986 19711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:01:20.415493 19711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:41965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:20.415751 19711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:20.416054 19711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:01:20.421079 19711 server_base.cc:1034] running on GCE node
W20260925 22:01:20.421094 19719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.421232 19716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:20.421098 19717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:20.421551 19711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:20.421861 19711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:20.423029 19711 hybrid_clock.cc:648] HybridClock initialized: now 1790373680423014 us; error 31 us; skew 500 ppm
I20260925 22:01:20.424458 19711 webserver.cc:460] Webserver started at http://127.9.112.67:39295/ using document root <none> and password file <none>
I20260925 22:01:20.424778 19711 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:20.424846 19711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:20.424975 19711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:20.426409 19711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "7d68d37f7a6347cd80af74477792c33a"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "d7dbcebc9e684d76563a178528471166"
server_key_iv: "867fd767d71a2a43031587be46456e91"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.426816 19711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "7d68d37f7a6347cd80af74477792c33a"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "d7dbcebc9e684d76563a178528471166"
server_key_iv: "867fd767d71a2a43031587be46456e91"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.428231 19711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:01:20.429309 19725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.429502 19711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:20.429610 19711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "7d68d37f7a6347cd80af74477792c33a"
format_stamp: "Formatted at 2026-09-25 22:01:20 on dist-test-slave-794d"
server_key: "d7dbcebc9e684d76563a178528471166"
server_key_iv: "867fd767d71a2a43031587be46456e91"
server_key_version: "encryptionkey@0"
I20260925 22:01:20.429687 19711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:20.434243 19711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:20.434713 19711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:20.434851 19711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:20.435067 19711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:20.435400 19711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:20.435438 19711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.435463 19711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:20.435477 19711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:20.441742 19711 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:39317
I20260925 22:01:20.441789 19838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:39317 every 8 connection(s)
I20260925 22:01:20.442140 19711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:01:20.445182 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.442934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:42233 (local address 127.9.112.126:41965)
0925 22:01:20.442981 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:20.442983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:20.443209 (+ 226us) server_negotiation.cc:407] Connection header received
0925 22:01:20.443740 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:20.443741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:20.443747 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:20.443807 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:20.444969 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:20.444970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:20.444970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:01:20.444974 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:20.444987 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:01:20.444991 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:20.445049 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:20.445102 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 22:01:20.445115 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:01:20.445354 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:20.442922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:41965 (local address 127.9.112.67:42233)
0925 22:01:20.443084 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:20.443118 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:20.443584 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:20.443878 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:20.443882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:20.443892 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:20.444820 (+ 928us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:01:20.444821 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:20.444850 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:01:20.444879 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:20.445049 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:20.445051 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:20.445053 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:01:20.445102 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:01:20.445151 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260925 22:01:20.445962 19839 heartbeater.cc:347] Connected to a master server at 127.9.112.126:41965
I20260925 22:01:20.446053 19839 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:20.446167 19839 heartbeater.cc:511] Master 127.9.112.126:41965 requested a full tablet report, sending...
I20260925 22:01:20.446506 19389 ts_manager.cc:194] Registered new tserver with Master: 7d68d37f7a6347cd80af74477792c33a (127.9.112.67:39317)
I20260925 22:01:20.448246 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19711
I20260925 22:01:20.448320 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:01:20.448596 9665 external_mini_cluster.cc:1428] Setting key fdf1e496b442675c7c103daf026d3b4c
I20260925 22:01:20.449391 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 22:01:20.451052 19439 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:01:20.450625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.9.112.126:41965)
0925 22:01:20.450690 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:20.450692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:20.450763 (+ 71us) server_negotiation.cc:407] Connection header received
0925 22:01:20.450808 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:20.450808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:20.450813 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:20.450874 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:20.450985 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39580: BlockingRecv error: recv got EOF from 127.0.0.1:39580 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:01:20.451093 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19449
I20260925 22:01:20.455621 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19580
I20260925 22:01:20.460091 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19711
I20260925 22:01:20.463838 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19358
2026-09-25T22:01:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (533 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:01:20 dist-test-slave-794d krb5kdc[19849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:01:20 dist-test-slave-794d krb5kdc[19849](info): set up 2 sockets
Sep 25 22:01:20 dist-test-slave-794d krb5kdc[19849](info): commencing operation
krb5kdc: starting...
W20260925 22:01:21.421022 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:01:21 dist-test-slave-794d krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:01:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:01:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:01:21.551591 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45917
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:21.630738 19862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:21.630992 19862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:21.631026 19862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:21.632471 19862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:01:21.632527 19862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:21.632546 19862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:21.632563 19862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:01:21.632577 19862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:01:21.634399 19862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:45917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:21.634699 19862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:21.634924 19862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:01:21.639935 19862 server_base.cc:1034] running on GCE node
W20260925 22:01:21.639884 19868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:21.639884 19867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:21.639884 19870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:21.640319 19862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:21.640671 19862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:21.641844 19862 hybrid_clock.cc:648] HybridClock initialized: now 1790373681641825 us; error 50 us; skew 500 ppm
Sep 25 22:01:21 dist-test-slave-794d krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:21.644572 19862 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:01:21.645103 19862 webserver.cc:460] Webserver started at http://127.9.112.126:41723/ using document root <none> and password file <none>
I20260925 22:01:21.645337 19862 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:21.645391 19862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:21.645519 19862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:21.646590 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "bf1f52f0fdb54f6189855f687f4ea2ab"
format_stamp: "Formatted at 2026-09-25 22:01:21 on dist-test-slave-794d"
server_key: "001a56e474ade4f150237a71ec3c34ba"
server_key_iv: "8c026ecc4ed962bee9c71befd313d738"
server_key_version: "encryptionkey@0"
I20260925 22:01:21.646934 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "bf1f52f0fdb54f6189855f687f4ea2ab"
format_stamp: "Formatted at 2026-09-25 22:01:21 on dist-test-slave-794d"
server_key: "001a56e474ade4f150237a71ec3c34ba"
server_key_iv: "8c026ecc4ed962bee9c71befd313d738"
server_key_version: "encryptionkey@0"
I20260925 22:01:21.648291 19862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:01:21.649194 19877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:21.649500 19862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:01:21.649600 19862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "bf1f52f0fdb54f6189855f687f4ea2ab"
format_stamp: "Formatted at 2026-09-25 22:01:21 on dist-test-slave-794d"
server_key: "001a56e474ade4f150237a71ec3c34ba"
server_key_iv: "8c026ecc4ed962bee9c71befd313d738"
server_key_version: "encryptionkey@0"
I20260925 22:01:21.649678 19862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:21.670933 19862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:21.673239 19862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:21.673403 19862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:21.677191 19862 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:45917
I20260925 22:01:21.677225 19929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:45917 every 8 connection(s)
I20260925 22:01:21.677567 19862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:01:21.678474 19930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:21.680852 19930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab: Bootstrap starting.
I20260925 22:01:21.681429 19930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:21.681694 19930 log.cc:824] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:21.682500 19930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab: No bootstrap required, opened a new log
I20260925 22:01:21.684167 19930 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45917 } }
I20260925 22:01:21.684317 19930 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:21.684368 19930 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1f52f0fdb54f6189855f687f4ea2ab, State: Initialized, Role: FOLLOWER
I20260925 22:01:21.684520 19930 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45917 } }
I20260925 22:01:21.684593 19930 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:01:21.684615 19930 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:01:21.684644 19930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:21.685493 19930 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45917 } }
I20260925 22:01:21.685577 19930 leader_election.cc:302] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf1f52f0fdb54f6189855f687f4ea2ab; no voters:
I20260925 22:01:21.685734 19930 leader_election.cc:288] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:01:21.685806 19933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:21.685966 19933 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [term 1 LEADER]: Becoming Leader. State: Replica: bf1f52f0fdb54f6189855f687f4ea2ab, State: Running, Role: LEADER
I20260925 22:01:21.685974 19930 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:01:21.686064 19933 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45917 } }
I20260925 22:01:21.686419 19934 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45917 } } }
I20260925 22:01:21.686441 19935 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf1f52f0fdb54f6189855f687f4ea2ab. Latest consensus state: current_term: 1 leader_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf1f52f0fdb54f6189855f687f4ea2ab" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45917 } } }
I20260925 22:01:21.686542 19935 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:21.686542 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19862
I20260925 22:01:21.686623 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:01:21.686812 9665 external_mini_cluster.cc:1428] Setting key 2a307cce5e87cedb7a09505bc6161e90
I20260925 22:01:21.687009 19934 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab [sys.catalog]: This master's current role is: LEADER
I20260925 22:01:21.687068 19937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:01:21.687593 19937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:01:21.689991 19937 catalog_manager.cc:1347] Generated new cluster ID: 7ff383f267104cddb36a981cf8bf6f89
I20260925 22:01:21.690048 19937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:01:21 dist-test-slave-794d krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:21.698674 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:21.687462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.9.112.126:45917)
0925 22:01:21.687670 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:21.687681 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:21.687714 (+ 33us) server_negotiation.cc:407] Connection header received
0925 22:01:21.687852 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:21.687860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:21.687916 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:21.688810 (+ 894us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:21.688925 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:21.690262 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:21.691282 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:21.691955 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:21.692079 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:21.694867 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:21.694873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:21.694873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:21.694887 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:21.696990 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:21.697428 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:21.697430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:21.697431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:21.697539 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:21.697733 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:21.697734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:21.697734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:21.698048 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:21.698192 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:21.698300 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 22:01:21.698439 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260925 22:01:21.711313 19937 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:01:21.712080 19937 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:01:21.718376 19937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab: Generated new TSK 0
I20260925 22:01:21.718614 19937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:01:21.751236 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45917
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:21.833087 19956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:21.833325 19956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:21.833359 19956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:21.834606 19956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:21.834652 19956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:21.834690 19956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:01:21.836206 19956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:21.836494 19956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:21.836706 19956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:01:21.841686 19956 server_base.cc:1034] running on GCE node
W20260925 22:01:21.841627 19962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:21.841627 19964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:21.841634 19961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:21.842049 19956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:21.842283 19956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:21.843473 19956 hybrid_clock.cc:648] HybridClock initialized: now 1790373681843463 us; error 30 us; skew 500 ppm
Sep 25 22:01:21 dist-test-slave-794d krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:21.846256 19956 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:01:21.846764 19956 webserver.cc:460] Webserver started at http://127.9.112.65:46345/ using document root <none> and password file <none>
I20260925 22:01:21.847002 19956 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:21.847056 19956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:21.847200 19956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:21.848315 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "709afd587ba041e7b6f5f94bc014f5eb"
format_stamp: "Formatted at 2026-09-25 22:01:21 on dist-test-slave-794d"
server_key: "fd95594c304026669bdc384ef0ca3a12"
server_key_iv: "a475e3ef8b7a3fef4330e2b9677da081"
server_key_version: "encryptionkey@0"
I20260925 22:01:21.848722 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "709afd587ba041e7b6f5f94bc014f5eb"
format_stamp: "Formatted at 2026-09-25 22:01:21 on dist-test-slave-794d"
server_key: "fd95594c304026669bdc384ef0ca3a12"
server_key_iv: "a475e3ef8b7a3fef4330e2b9677da081"
server_key_version: "encryptionkey@0"
I20260925 22:01:21.850143 19956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:01:21.851009 19971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:21.851205 19956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:01:21.851305 19956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "709afd587ba041e7b6f5f94bc014f5eb"
format_stamp: "Formatted at 2026-09-25 22:01:21 on dist-test-slave-794d"
server_key: "fd95594c304026669bdc384ef0ca3a12"
server_key_iv: "a475e3ef8b7a3fef4330e2b9677da081"
server_key_version: "encryptionkey@0"
I20260925 22:01:21.851382 19956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:21.868501 19956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:21.870759 19956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:21.870918 19956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:21.871163 19956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:21.871469 19956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:21.871502 19956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:21.871531 19956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:21.871556 19956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:21.879681 19956 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:44673
I20260925 22:01:21.879870 20084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:44673 every 8 connection(s)
I20260925 22:01:21.880185 19956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:01:21 dist-test-slave-794d krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373681, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:21.886991 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 19956
I20260925 22:01:21.887109 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:01:21.887317 9665 external_mini_cluster.cc:1428] Setting key d7bf73661a6a0c4cb1f61264dae01038
I20260925 22:01:21.890769 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:21.881160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:48855 (local address 127.9.112.126:45917)
0925 22:01:21.881236 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:21.881238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:21.881456 (+ 218us) server_negotiation.cc:407] Connection header received
0925 22:01:21.881971 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:21.881973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:21.881983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:21.882048 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:21.883050 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:21.883594 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:21.884804 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:21.885065 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:21.885132 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:21.887599 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:21.887601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:21.887601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:21.887609 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:21.889264 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:21.889697 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:21.889698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:21.889699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:21.889764 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:21.890095 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:21.890096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:21.890096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:21.890241 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:21.890305 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:21.890600 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 22:01:21.890686 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:01:21.890906 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:21.881119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45917 (local address 127.9.112.65:48855)
0925 22:01:21.881302 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:21.881354 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:21.881848 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:21.882108 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:21.882112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:21.882719 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:21.882931 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:21.882936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:21.883711 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:21.883712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:21.884711 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:21.884714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:21.885143 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:21.885144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:21.885252 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:21.885805 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:21.885814 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:21.887509 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:21.889368 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:21.889376 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:21.889379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:21.889622 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:21.889851 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:21.889851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:21.889851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:21.890011 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:21.890311 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:21.890317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:21.890461 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 22:01:21.890554 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 22:01:21.890687 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260925 22:01:21.891454 20085 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45917
I20260925 22:01:21.891554 20085 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:21.891732 20085 heartbeater.cc:511] Master 127.9.112.126:45917 requested a full tablet report, sending...
I20260925 22:01:21.892409 19894 ts_manager.cc:194] Registered new tserver with Master: 709afd587ba041e7b6f5f94bc014f5eb (127.9.112.65:44673)
I20260925 22:01:21.893360 19894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:48855
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:01:21.917003 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45917
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:21.998759 20090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:21.998989 20090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:21.999013 20090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:22.000214 20090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:22.000272 20090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:22.000300 20090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:01:22.002184 20090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:22.002507 20090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:22.002739 20090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:01:22.008039 20096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:22.008051 20095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:22.008039 20098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:22.008213 20090 server_base.cc:1034] running on GCE node
I20260925 22:01:22.008550 20090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:22.008805 20090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:22.009970 20090 hybrid_clock.cc:648] HybridClock initialized: now 1790373682009954 us; error 29 us; skew 500 ppm
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373682, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:22.012830 20090 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:01:22.013283 20090 webserver.cc:460] Webserver started at http://127.9.112.66:33451/ using document root <none> and password file <none>
I20260925 22:01:22.013506 20090 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:22.013552 20090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:22.013660 20090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:22.014688 20090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "a7d35464753d4a68bf78ba260e9cea0b"
format_stamp: "Formatted at 2026-09-25 22:01:22 on dist-test-slave-794d"
server_key: "dd0571190de65bfc667d8febeccba825"
server_key_iv: "da33795d38a4825d82aafe244dbe2578"
server_key_version: "encryptionkey@0"
I20260925 22:01:22.015004 20090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "a7d35464753d4a68bf78ba260e9cea0b"
format_stamp: "Formatted at 2026-09-25 22:01:22 on dist-test-slave-794d"
server_key: "dd0571190de65bfc667d8febeccba825"
server_key_iv: "da33795d38a4825d82aafe244dbe2578"
server_key_version: "encryptionkey@0"
I20260925 22:01:22.016275 20090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:01:22.017083 20105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.017238 20090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:01:22.017313 20090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "a7d35464753d4a68bf78ba260e9cea0b"
format_stamp: "Formatted at 2026-09-25 22:01:22 on dist-test-slave-794d"
server_key: "dd0571190de65bfc667d8febeccba825"
server_key_iv: "da33795d38a4825d82aafe244dbe2578"
server_key_version: "encryptionkey@0"
I20260925 22:01:22.017383 20090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:22.037222 20090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:22.039399 20090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:22.039516 20090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:22.039726 20090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:22.040024 20090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:22.040056 20090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.040083 20090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:22.040108 20090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.045661 20090 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:40233
I20260925 22:01:22.045718 20218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:40233 every 8 connection(s)
I20260925 22:01:22.046029 20090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:01:22.052203 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 20090
I20260925 22:01:22.052330 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373682, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:22.052537 9665 external_mini_cluster.cc:1428] Setting key f72f5b3327cc71d64c57a5c1c6e1820f
I20260925 22:01:22.055739 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.046708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:53933 (local address 127.9.112.126:45917)
0925 22:01:22.046783 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.046785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.046953 (+ 168us) server_negotiation.cc:407] Connection header received
0925 22:01:22.047480 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.047481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.047490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.047527 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:22.048548 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.049071 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.050103 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.050315 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.050384 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.052860 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:22.052862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:22.052862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:22.052873 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:22.054292 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:22.054738 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:22.054739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:22.054740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:22.054806 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:22.055063 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:22.055064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:22.055064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:22.055204 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:22.055272 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.055554 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.055642 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:01:22.055956 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.046676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45917 (local address 127.9.112.66:53933)
0925 22:01:22.046829 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.046866 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.047385 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.047587 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.047590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.048169 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:22.048433 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.048439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.049154 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.049155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.049970 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.049972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.050437 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.050439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.050558 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.051133 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:22.051142 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:22.052764 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:22.054377 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:22.054384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:22.054387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:22.054653 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:22.054872 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:22.054873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:22.054873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:22.055006 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:22.055276 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:22.055281 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:22.055422 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.055508 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.055670 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260925 22:01:22.056599 20219 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45917
I20260925 22:01:22.056699 20219 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:22.056890 20219 heartbeater.cc:511] Master 127.9.112.126:45917 requested a full tablet report, sending...
I20260925 22:01:22.057358 19894 ts_manager.cc:194] Registered new tserver with Master: a7d35464753d4a68bf78ba260e9cea0b (127.9.112.66:40233)
I20260925 22:01:22.057734 19894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:53933
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:01:22.079582 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45917
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:01:22.162496 20224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:01:22.162729 20224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:01:22.162755 20224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:01:22.163975 20224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:01:22.164026 20224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:01:22.164055 20224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:01:22.165968 20224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:01:22.166229 20224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:01:22.166478 20224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:01:22.171852 20224 server_base.cc:1034] running on GCE node
W20260925 22:01:22.171787 20232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:22.171762 20229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:01:22.171774 20230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:01:22.172197 20224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:01:22.172472 20224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:01:22.173619 20224 hybrid_clock.cc:648] HybridClock initialized: now 1790373682173577 us; error 58 us; skew 500 ppm
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373682, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:01:22.176424 20224 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:01:22.176916 20224 webserver.cc:460] Webserver started at http://127.9.112.67:35695/ using document root <none> and password file <none>
I20260925 22:01:22.177119 20224 fs_manager.cc:359] Metadata directory not provided
I20260925 22:01:22.177167 20224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:01:22.177269 20224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:01:22.178334 20224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "6408eb6ec8e948869deb2f69375fa3d6"
format_stamp: "Formatted at 2026-09-25 22:01:22 on dist-test-slave-794d"
server_key: "83a4b21563433375389a92c8f27d61b4"
server_key_iv: "0bd09ce19fd5f767f187cdd45dcf4a4b"
server_key_version: "encryptionkey@0"
I20260925 22:01:22.178653 20224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "6408eb6ec8e948869deb2f69375fa3d6"
format_stamp: "Formatted at 2026-09-25 22:01:22 on dist-test-slave-794d"
server_key: "83a4b21563433375389a92c8f27d61b4"
server_key_iv: "0bd09ce19fd5f767f187cdd45dcf4a4b"
server_key_version: "encryptionkey@0"
I20260925 22:01:22.180073 20224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:01:22.180900 20239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.181049 20224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.181134 20224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "6408eb6ec8e948869deb2f69375fa3d6"
format_stamp: "Formatted at 2026-09-25 22:01:22 on dist-test-slave-794d"
server_key: "83a4b21563433375389a92c8f27d61b4"
server_key_iv: "0bd09ce19fd5f767f187cdd45dcf4a4b"
server_key_version: "encryptionkey@0"
I20260925 22:01:22.181195 20224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:01:22.214152 20224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:01:22.216892 20224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:01:22.217046 20224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:01:22.217264 20224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:01:22.217599 20224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:01:22.217633 20224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.217664 20224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:01:22.217686 20224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:01:22.224044 20224 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:34585
I20260925 22:01:22.224103 20352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:34585 every 8 connection(s)
I20260925 22:01:22.224475 20224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:01:22.224993 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 20224
I20260925 22:01:22.225077 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:01:22.225260 9665 external_mini_cluster.cc:1428] Setting key a98e983f4969195f12b0b8e2d8574b9e
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373682, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:01:22.234457 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.225261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:50645 (local address 127.9.112.126:45917)
0925 22:01:22.225310 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.225312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.225631 (+ 319us) server_negotiation.cc:407] Connection header received
0925 22:01:22.226232 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.226233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.226245 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.226302 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:22.227286 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.227784 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.228812 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.229019 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.229076 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.231524 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:22.231527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:22.231527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:22.231538 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:22.233041 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:22.233498 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:22.233500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:22.233500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:22.233566 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:22.233792 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:22.233793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:22.233793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:22.233907 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:22.233962 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.234277 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.234362 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:01:22.234649 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.225248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45917 (local address 127.9.112.67:50645)
0925 22:01:22.225425 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.225474 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.226140 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.226385 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.226390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.226977 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:01:22.227184 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.227189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.227870 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.227872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.228729 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.228732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.229089 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.229090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.229221 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.229896 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:01:22.229905 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:01:22.231419 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:01:22.233152 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:22.233171 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:22.233174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:22.233424 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:22.233624 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:01:22.233625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:01:22.233625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:01:22.233736 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:01:22.233961 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:01:22.233968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:01:22.234111 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.234243 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.234417 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260925 22:01:22.235313 20353 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45917
I20260925 22:01:22.235432 20353 heartbeater.cc:464] Registering TS with master...
I20260925 22:01:22.235612 20353 heartbeater.cc:511] Master 127.9.112.126:45917 requested a full tablet report, sending...
I20260925 22:01:22.236056 19894 ts_manager.cc:194] Registered new tserver with Master: 6408eb6ec8e948869deb2f69375fa3d6 (127.9.112.67:34585)
I20260925 22:01:22.236466 19894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:50645
I20260925 22:01:22.237157 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:01:22.244333 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.238422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47616 (local address 127.9.112.126:45917)
0925 22:01:22.238516 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.238518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.238593 (+ 75us) server_negotiation.cc:407] Connection header received
0925 22:01:22.238658 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.238659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.238668 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.238720 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:01:22.239248 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.239740 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.240467 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.240671 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.240725 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.242069 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:01:22.242070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:01:22.242071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:01:22.242078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:01:22.243289 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:22.243597 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:22.243598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:22.243598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:22.243649 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:01:22.243814 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:01:22.243815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:01:22.243815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:01:22.243968 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:01:22.244053 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.244160 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.244237 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 22:01:22.246253 19893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:01:22.247886 19893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:01:22.254846 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.250396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:40233 (local address 127.0.0.1:34560)
0925 22:01:22.250523 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.250565 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.250632 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.251138 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.251139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.251141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:22.251169 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.251170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.252550 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.252551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.253689 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.253692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.254504 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.254505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.254593 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.254608 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.254675 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.254742 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":37,"thread_start_us":84,"threads_started":1}
I20260925 22:01:22.255096 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.250686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.9.112.65:44673)
0925 22:01:22.250879 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.250884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.250894 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:01:22.250934 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.250937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.250952 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.251029 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:22.251208 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.252145 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.253804 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.254516 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.254903 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.254962 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.254989 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.255031 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260925 22:01:22.254863 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.250404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44673 (local address 127.0.0.1:39208)
0925 22:01:22.250680 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.250691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.250760 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.251051 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.251055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.251057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:22.251129 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.251131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.252232 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.252233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.253689 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.253692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.254616 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.254617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.254687 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.254692 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.254720 (+ 28us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.254763 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":6,"thread_start_us":98,"threads_started":1}
I20260925 22:01:22.255185 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.250613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.9.112.66:40233)
0925 22:01:22.250946 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.250951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.250963 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:01:22.251006 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.251008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.251025 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.251149 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:22.251233 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.252451 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.253781 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.254426 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.254492 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.254556 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.254905 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.254951 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":80,"threads_started":1}
I20260925 22:01:22.255343 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.250404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:34585 (local address 127.0.0.1:44430)
0925 22:01:22.250846 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.250854 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.250892 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.251476 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.251476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.251479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:22.251518 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.251519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.252888 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.252889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.253835 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.253837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.254720 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.254720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.254772 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.254776 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.254826 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.254861 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260925 22:01:22.255630 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.250785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.9.112.67:34585)
0925 22:01:22.251006 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.251009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.251019 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:01:22.251303 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.251306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.251322 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.251475 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:22.251584 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.252797 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.253929 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.254624 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.254687 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.254743 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.254838 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.254875 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
I20260925 22:01:22.255918 20019 tablet_service.cc:1467] Processing CreateTablet for tablet f47f2c690c674e0895d70af1e3fe0ced (DEFAULT_TABLE table=test-table [id=b225513fc67a41dcafa776af7484ff32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:22.255919 20153 tablet_service.cc:1467] Processing CreateTablet for tablet f47f2c690c674e0895d70af1e3fe0ced (DEFAULT_TABLE table=test-table [id=b225513fc67a41dcafa776af7484ff32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:22.256278 20019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f2c690c674e0895d70af1e3fe0ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:22.256278 20153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f2c690c674e0895d70af1e3fe0ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:22.256245 20287 tablet_service.cc:1467] Processing CreateTablet for tablet f47f2c690c674e0895d70af1e3fe0ced (DEFAULT_TABLE table=test-table [id=b225513fc67a41dcafa776af7484ff32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:01:22.256560 20287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f2c690c674e0895d70af1e3fe0ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:01:22.259053 20370 tablet_bootstrap.cc:492] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Bootstrap starting.
I20260925 22:01:22.259482 20371 tablet_bootstrap.cc:492] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: Bootstrap starting.
I20260925 22:01:22.259838 20370 tablet_bootstrap.cc:654] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:22.260064 20371 tablet_bootstrap.cc:654] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:22.260416 20370 log.cc:824] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:22.260476 20372 tablet_bootstrap.cc:492] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: Bootstrap starting.
I20260925 22:01:22.260636 20371 log.cc:824] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:22.261054 20372 tablet_bootstrap.cc:654] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: Neither blocks nor log segments found. Creating new log.
I20260925 22:01:22.261201 20370 tablet_bootstrap.cc:492] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: No bootstrap required, opened a new log
I20260925 22:01:22.261261 20370 ts_tablet_manager.cc:1397] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260925 22:01:22.261473 20371 tablet_bootstrap.cc:492] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: No bootstrap required, opened a new log
I20260925 22:01:22.261541 20371 ts_tablet_manager.cc:1397] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:01:22.261586 20372 log.cc:824] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:01:22.262512 20372 tablet_bootstrap.cc:492] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: No bootstrap required, opened a new log
I20260925 22:01:22.262578 20372 ts_tablet_manager.cc:1397] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:01:22.262920 20370 raft_consensus.cc:348] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.263051 20370 raft_consensus.cc:374] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:22.263083 20370 raft_consensus.cc:729] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6408eb6ec8e948869deb2f69375fa3d6, State: Initialized, Role: FOLLOWER
I20260925 22:01:22.263192 20370 consensus_queue.cc:260] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.263466 20353 heartbeater.cc:503] Master 127.9.112.126:45917 was elected leader, sending a full tablet report...
I20260925 22:01:22.263468 20370 ts_tablet_manager.cc:1428] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 22:01:22.263401 20371 raft_consensus.cc:348] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.263873 20371 raft_consensus.cc:374] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:22.263916 20371 raft_consensus.cc:729] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709afd587ba041e7b6f5f94bc014f5eb, State: Initialized, Role: FOLLOWER
I20260925 22:01:22.264024 20371 consensus_queue.cc:260] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.264283 20371 ts_tablet_manager.cc:1428] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:01:22.264446 20085 heartbeater.cc:503] Master 127.9.112.126:45917 was elected leader, sending a full tablet report...
I20260925 22:01:22.264465 20372 raft_consensus.cc:348] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.264607 20372 raft_consensus.cc:374] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:01:22.264648 20372 raft_consensus.cc:729] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d35464753d4a68bf78ba260e9cea0b, State: Initialized, Role: FOLLOWER
I20260925 22:01:22.264760 20372 consensus_queue.cc:260] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.265017 20372 ts_tablet_manager.cc:1428] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:01:22.265028 20219 heartbeater.cc:503] Master 127.9.112.126:45917 was elected leader, sending a full tablet report...
W20260925 22:01:22.296634 20220 tablet.cc:2369] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:01:22.381163 20086 tablet.cc:2369] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:01:22.455459 20376 raft_consensus.cc:482] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:01:22.455583 20376 raft_consensus.cc:504] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.455963 20376 leader_election.cc:288] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 709afd587ba041e7b6f5f94bc014f5eb (127.9.112.65:44673), a7d35464753d4a68bf78ba260e9cea0b (127.9.112.66:40233)
I20260925 22:01:22.458765 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.456266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:37395 (local address 127.9.112.66:40233)
0925 22:01:22.456304 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.456305 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.456315 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:01:22.456372 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.456373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.456383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.456436 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:22.456564 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.457075 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.458043 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.458431 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.458496 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.458506 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.458654 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.458690 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:01:22.458823 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.456185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:40233 (local address 127.9.112.67:37395)
0925 22:01:22.456241 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.456253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.456302 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.456440 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.456441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.456443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:22.456508 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.456509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.457165 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.457166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.457953 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.457955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.458506 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.458506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.458593 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.458596 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.458641 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.458688 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":24}
I20260925 22:01:22.458806 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.456227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:60769 (local address 127.9.112.65:44673)
0925 22:01:22.456275 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.456277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.456433 (+ 156us) server_negotiation.cc:407] Connection header received
0925 22:01:22.456503 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.456504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.456513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.456558 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:01:22.456680 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.457179 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.458124 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.458489 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.458560 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.458569 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.458699 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.458737 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:01:22.459024 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.456192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44673 (local address 127.9.112.67:60769)
0925 22:01:22.456367 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.456376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:01:22.456442 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:01:22.456560 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:01:22.456560 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:01:22.456562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:01:22.456628 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.456630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.457261 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.457262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.458040 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:01:22.458042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.458571 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.458573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:01:22.458640 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.458644 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:01:22.458688 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:01:22.458728 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":15,"thread_start_us":87,"threads_started":1}
I20260925 22:01:22.459165 20173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f2c690c674e0895d70af1e3fe0ced" candidate_uuid: "6408eb6ec8e948869deb2f69375fa3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d35464753d4a68bf78ba260e9cea0b" is_pre_election: true
I20260925 22:01:22.459308 20173 raft_consensus.cc:2454] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6408eb6ec8e948869deb2f69375fa3d6 in term 0.
I20260925 22:01:22.459549 20240 leader_election.cc:302] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6408eb6ec8e948869deb2f69375fa3d6, a7d35464753d4a68bf78ba260e9cea0b; no voters:
I20260925 22:01:22.459633 20039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f2c690c674e0895d70af1e3fe0ced" candidate_uuid: "6408eb6ec8e948869deb2f69375fa3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "709afd587ba041e7b6f5f94bc014f5eb" is_pre_election: true
I20260925 22:01:22.459774 20039 raft_consensus.cc:2454] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6408eb6ec8e948869deb2f69375fa3d6 in term 0.
I20260925 22:01:22.460289 20376 raft_consensus.cc:2781] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:01:22.460376 20376 raft_consensus.cc:482] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:01:22.460412 20376 raft_consensus.cc:3037] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:22.461086 20376 raft_consensus.cc:504] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.461210 20376 leader_election.cc:288] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 709afd587ba041e7b6f5f94bc014f5eb (127.9.112.65:44673), a7d35464753d4a68bf78ba260e9cea0b (127.9.112.66:40233)
I20260925 22:01:22.461351 20039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f2c690c674e0895d70af1e3fe0ced" candidate_uuid: "6408eb6ec8e948869deb2f69375fa3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "709afd587ba041e7b6f5f94bc014f5eb"
I20260925 22:01:22.461393 20173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f2c690c674e0895d70af1e3fe0ced" candidate_uuid: "6408eb6ec8e948869deb2f69375fa3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d35464753d4a68bf78ba260e9cea0b"
I20260925 22:01:22.461440 20039 raft_consensus.cc:3037] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:22.461460 20173 raft_consensus.cc:3037] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:01:22.462117 20039 raft_consensus.cc:2454] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6408eb6ec8e948869deb2f69375fa3d6 in term 1.
I20260925 22:01:22.462128 20173 raft_consensus.cc:2454] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6408eb6ec8e948869deb2f69375fa3d6 in term 1.
I20260925 22:01:22.462283 20242 leader_election.cc:302] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6408eb6ec8e948869deb2f69375fa3d6, 709afd587ba041e7b6f5f94bc014f5eb; no voters:
I20260925 22:01:22.462369 20376 raft_consensus.cc:2781] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:01:22.462504 20376 raft_consensus.cc:686] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6408eb6ec8e948869deb2f69375fa3d6, State: Running, Role: LEADER
I20260925 22:01:22.462586 20376 consensus_queue.cc:237] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } }
I20260925 22:01:22.463407 19893 catalog_manager.cc:5495] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6408eb6ec8e948869deb2f69375fa3d6 (127.9.112.67). New cstate: current_term: 1 leader_uuid: "6408eb6ec8e948869deb2f69375fa3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6408eb6ec8e948869deb2f69375fa3d6" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 34585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 } health_report { overall_health: UNKNOWN } } }
W20260925 22:01:22.475080 20354 tablet.cc:2369] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:01:22.482815 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.480020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.9.112.67:34585)
0925 22:01:22.480078 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:01:22.480080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:01:22.480089 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:01:22.480151 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:01:22.480153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:01:22.480170 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:01:22.480231 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:01:22.480429 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.480958 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.481891 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:01:22.482084 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:01:22.482145 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:01:22.482355 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:01:22.482467 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:01:22.482540 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:01:22.482678 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:01:22.482708 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:01:22.484877 20039 raft_consensus.cc:1261] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 1 FOLLOWER]: Refusing update from remote peer 6408eb6ec8e948869deb2f69375fa3d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:22.484946 20173 raft_consensus.cc:1261] T f47f2c690c674e0895d70af1e3fe0ced P a7d35464753d4a68bf78ba260e9cea0b [term 1 FOLLOWER]: Refusing update from remote peer 6408eb6ec8e948869deb2f69375fa3d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:01:22.485311 20376 consensus_queue.cc:1035] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "709afd587ba041e7b6f5f94bc014f5eb" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:22.485471 20380 consensus_queue.cc:1035] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d35464753d4a68bf78ba260e9cea0b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:01:22.487980 20383 mvcc.cc:205] Tried to move back new op lower bound from 7333370603454869504 to 7333370603367268352. Current Snapshot: MvccSnapshot[applied={T|T < 7333370603454869504}]
I20260925 22:01:22.488617 20384 mvcc.cc:205] Tried to move back new op lower bound from 7333370603454869504 to 7333370603367268352. Current Snapshot: MvccSnapshot[applied={T|T < 7333370603454869504}]
I20260925 22:01:22.489082 20385 mvcc.cc:205] Tried to move back new op lower bound from 7333370603454869504 to 7333370603367268352. Current Snapshot: MvccSnapshot[applied={T|T < 7333370603454869504}]
I20260925 22:01:22.490573 19894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:22.490682 19894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:01:22.491482 19894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf1f52f0fdb54f6189855f687f4ea2ab: Sending DeleteTablet for 3 replicas of tablet f47f2c690c674e0895d70af1e3fe0ced
I20260925 22:01:22.491807 20019 tablet_service.cc:1514] Processing DeleteTablet for tablet f47f2c690c674e0895d70af1e3fe0ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:22 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:39208
I20260925 22:01:22.491847 20153 tablet_service.cc:1514] Processing DeleteTablet for tablet f47f2c690c674e0895d70af1e3fe0ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:22 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:34560
I20260925 22:01:22.492134 20287 tablet_service.cc:1514] Processing DeleteTablet for tablet f47f2c690c674e0895d70af1e3fe0ced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:01:22 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:44430
I20260925 22:01:22.492237 20392 tablet_replica.cc:330] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb: stopping tablet replica
I20260925 22:01:22.492364 20392 raft_consensus.cc:2229] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:01:22.492414 20394 tablet_replica.cc:330] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: stopping tablet replica
I20260925 22:01:22.492502 20392 raft_consensus.cc:2258] T f47f2c690c674e0895d70af1e3fe0ced P 709afd587ba041e7b6f5f94bc014f5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:22.492530 20394 raft_consensus.cc:2229] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:01:22.492658 20394 raft_consensus.cc:2258] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:01:22.492748 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19956
I20260925 22:01:22.493198 20394 ts_tablet_manager.cc:1905] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:01:22.495005 20394 ts_tablet_manager.cc:1918] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:01:22.495088 20394 log.cc:1196] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/f47f2c690c674e0895d70af1e3fe0ced
I20260925 22:01:22.495348 20394 ts_tablet_manager.cc:1939] T f47f2c690c674e0895d70af1e3fe0ced P 6408eb6ec8e948869deb2f69375fa3d6: Deleting consensus metadata
I20260925 22:01:22.495939 19881 catalog_manager.cc:4841] TS 6408eb6ec8e948869deb2f69375fa3d6 (127.9.112.67:34585): tablet f47f2c690c674e0895d70af1e3fe0ced (table test-table [id=b225513fc67a41dcafa776af7484ff32]) successfully deleted
W20260925 22:01:22.498543 19880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:44673 (error 108)
I20260925 22:01:22.498688 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 20090
I20260925 22:01:22.504503 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.499716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44673 (local address 127.0.0.1:39222)
0925 22:01:22.504385 (+ 4669us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.504452 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:44673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4616}
I20260925 22:01:22.504602 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:01:22.504446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:40233 (local address 127.0.0.1:34568)
0925 22:01:22.504494 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 22:01:22.504515 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.66:40233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:01:22.504779 19878 catalog_manager.cc:4584] TS a7d35464753d4a68bf78ba260e9cea0b (127.9.112.66:40233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f47f2c690c674e0895d70af1e3fe0ced: Network error: Client connection negotiation failed: client connection to 127.9.112.66:40233: connect: Connection refused (error 111)
I20260925 22:01:22.505403 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 20224
I20260925 22:01:22.509549 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 19862
2026-09-25T22:01:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2046 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[20398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[20398](info): set up 2 sockets
Sep 25 22:01:22 dist-test-slave-794d krb5kdc[20398](info): commencing operation
krb5kdc: starting...
W20260925 22:01:23.490645 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:01:23 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:01:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:01:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/rangeradmin_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/rangeradmin_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/rangerlookup_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/rangerlookup_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
I20260925 22:01:23.667563 9665 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQ0dh6G/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/postgres -l logfile start
2026-09-25 22:01:26.446 UTC [20428] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 22:01:26.447 UTC [20428] LOG: listening on IPv4 address "127.9.112.84", port 48163
2026-09-25 22:01:26.451 UTC [20428] LOG: listening on Unix socket "/tmp/.s.PGSQL.48163"
2026-09-25 22:01:26.480 UTC [20431] LOG: database system was shut down at 2026-09-25 22:01:24 UTC
2026-09-25 22:01:26.484 UTC [20428] LOG: database system is ready to accept connections
I20260925 22:01:27.353073 9665 mini_postgres.cc:96] Postgres bound to 48163
2026-09-25 22:01:27.358 UTC [20439] FATAL: database "slave" does not exist
127.9.112.84:48163 - accepting connections
I20260925 22:01:27.359404 9665 mini_ranger.cc:161] Starting Ranger...
I20260925 22:01:27.374058 9665 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 22:01:28.006910 9665 mini_ranger.cc:87] Created ranger Postgres database
I20260925 22:01:28.006981 9665 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-admin
2026-09-25 22:01:28,283 [I] DB FLAVOR :POSTGRES
2026-09-25 22:01:28,283 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:01:28,283 [I] Checking connection..
2026-09-25 22:01:28,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 22:01:28,556 [I] Checking connection passed.
2026-09-25 22:01:28,557 [I] --------- Verifying version history table ---------
2026-09-25 22:01:28,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:01:28,809 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:01:28,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:01:29,054 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:01:29,054 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 22:01:29,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 22:01:29.291 UTC [20503] WARNING: there is no transaction in progress
2026-09-25 22:01:29,297 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 22:01:29,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:01:29,564 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 22:01:29,564 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:01:29,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 22:01:29,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 22:01:30,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-794d.c.gcp-upstream.internal','N') ;"
2026-09-25 22:01:30,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:01:30,559 [I] Table x_portal_user does not exist in database ranger
2026-09-25 22:01:30,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:01:30,832 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 22:01:30,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:01:31,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:01:31,343 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 22:01:31,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 22:01:32.454 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.474 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.494 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.512 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.531 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.552 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.615 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.625 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.643 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.657 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.671 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.689 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.700 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.709 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.718 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:32.727 UTC [20611] WARNING: there is no transaction in progress
2026-09-25 22:01:33,526 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 22:01:33,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 22:01:33,782 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 22:01:33,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:01:34,035 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 22:01:34,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:01:34,286 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 22:01:34,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:01:34,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:01:34,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-794d.c.gcp-upstream.internal';"
2026-09-25 22:01:35,042 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 22:01:35,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 22:01:35,289 [I] DB_PATCHES have already been applied
I20260925 22:01:35.295761 9665 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQ0dh6G/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 22:01:35.295835 9665 mini_ranger.cc:193] Using host: 127.9.112.84
I20260925 22:01:35.298940 9665 mini_ranger.cc:227] Ranger admin URL: http://127.9.112.84:35085
Sep 25, 2026 10:01:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:01:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:01:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 10:01:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 10:01:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 10:01:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 10:01:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 22:01:36.316044 9665 mini_ranger.cc:160] Time spent starting Ranger: real 8.957s user 0.000s sys 0.009s
Sep 25, 2026 10:01:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 10:01:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 10:01:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 10:01:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 10:01:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 22:01:58.358935 9665 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/rangerkms_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/rangerkms_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.112.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.112.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 22:01:58 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373718, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 22:01:58.440166 9665 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 22:01:58.449088 9665 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 22:01:59.487033 9665 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 22:01:59.487118 9665 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms
I20260925 22:01:59.487135 9665 mini_ranger_kms.cc:227] Using postgres at 127.9.112.84:48163
2026-09-25 22:01:59,655 [I] DB FLAVOR :POSTGRES
2026-09-25 22:01:59,655 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:01:59,655 [I] Checking connection
2026-09-25 22:01:59,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 22:01:59,924 [I] connection success
2026-09-25 22:01:59,924 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 22:01:59,924 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 22:01:59,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 22:02:00,192 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 22:02:00,193 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:02:00,193 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 22:02:00,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:48163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 22:02:00,474 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 22:02:00.924256 9665 mini_ranger_kms.cc:310] Created kms service
I20260925 22:02:01.169766 9665 mini_ranger_kms.cc:326] Created kudu user
I20260925 22:02:01.210601 9665 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 22:02:01 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373718, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:01.622612 9665 mini_ranger_kms.cc:384] Added ranger policy
I20260925 22:02:01.622869 9665 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 22:02:01.622898 9665 mini_ranger_kms.cc:240] Using host: 127.9.112.84
I20260925 22:02:01.626362 9665 mini_ranger_kms.cc:276] Ranger KMS PID: 20791
I20260925 22:02:01.626500 9665 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.112.84:35737
Sep 25, 2026 10:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 10:02:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 10:02:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 10:02:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 22:02:03.644100 9665 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.204s user 0.000s sys 0.151s
I20260925 22:02:03.644199 9665 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 22:02:03.644269 9665 mini_ranger_kms.cc:401] 127.9.112.84:35737/kms/v1/keys
Sep 25 22:02:03 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790373723, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.112.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 22:02:04 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373723, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.112.84@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
Sep 25 22:02:04 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373723, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.112.84@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
Sep 25 22:02:07 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:02:07.136309 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:33159
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:33159
--ranger_config_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:35737/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:07.221876 20842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:07.222098 20842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:07.222121 20842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:07.223439 20842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:02:07.223501 20842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:07.223516 20842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:07.223527 20842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:02:07.223541 20842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:02:07.225432 20842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.112.84:35737/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:33159
--ranger_config_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:33159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:07.225679 20842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:07.225929 20842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:07.232319 20848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:07.232441 20842 server_base.cc:1034] running on GCE node
W20260925 22:02:07.232298 20847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:07.232410 20850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:07.232887 20842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:07.233242 20842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:07.234417 20842 hybrid_clock.cc:648] HybridClock initialized: now 1790373727234417 us; error 55 us; skew 500 ppm
Sep 25 22:02:07 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:07.237557 20842 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:02:07.238046 20842 webserver.cc:460] Webserver started at http://127.9.112.126:40151/ using document root <none> and password file <none>
I20260925 22:02:07.238270 20842 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:07.238308 20842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:07.238476 20842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:07 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:07.299136 20842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "e87689ec013e4e3590c46d848cca1dcf"
format_stamp: "Formatted at 2026-09-25 22:02:07 on dist-test-slave-794d"
server_key: "f5076359730d5aaaa84d7646456765a1"
server_key_iv: "9526379c37ccd289b749769c1a9e4822"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:07.299527 20842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "e87689ec013e4e3590c46d848cca1dcf"
format_stamp: "Formatted at 2026-09-25 22:02:07 on dist-test-slave-794d"
server_key: "f5076359730d5aaaa84d7646456765a1"
server_key_iv: "9526379c37ccd289b749769c1a9e4822"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:07.300895 20842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:02:07.312729 20857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:07.312991 20842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:07.313081 20842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "e87689ec013e4e3590c46d848cca1dcf"
format_stamp: "Formatted at 2026-09-25 22:02:07 on dist-test-slave-794d"
server_key: "f5076359730d5aaaa84d7646456765a1"
server_key_iv: "9526379c37ccd289b749769c1a9e4822"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:07.313179 20842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:07.328442 20842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:07.330341 20842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:07.330483 20842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:07.335237 20842 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:33159
I20260925 22:02:07.335255 20909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:33159 every 8 connection(s)
I20260925 22:02:07.335670 20842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:02:07.336305 20910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:02:07.338510 20910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf: Bootstrap starting.
I20260925 22:02:07.339462 20910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf: Neither blocks nor log segments found. Creating new log.
I20260925 22:02:07.339709 20910 log.cc:824] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf: Log is configured to *not* fsync() on all Append() calls
I20260925 22:02:07.340330 20910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf: No bootstrap required, opened a new log
I20260925 22:02:07.342440 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 20842
I20260925 22:02:07.342540 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:02:07.342509 20910 raft_consensus.cc:348] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87689ec013e4e3590c46d848cca1dcf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33159 } }
I20260925 22:02:07.342700 20910 raft_consensus.cc:374] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:02:07.342731 9665 external_mini_cluster.cc:1428] Setting key df2d49735927708082675c6c6f4d4f8b
I20260925 22:02:07.342774 20910 raft_consensus.cc:729] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87689ec013e4e3590c46d848cca1dcf, State: Initialized, Role: FOLLOWER
I20260925 22:02:07.342972 20910 consensus_queue.cc:260] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87689ec013e4e3590c46d848cca1dcf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33159 } }
I20260925 22:02:07.343062 20910 raft_consensus.cc:388] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:02:07.343093 20910 raft_consensus.cc:482] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:02:07.343134 20910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:02:07.343851 20910 raft_consensus.cc:504] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87689ec013e4e3590c46d848cca1dcf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33159 } }
I20260925 22:02:07.344030 20910 leader_election.cc:302] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e87689ec013e4e3590c46d848cca1dcf; no voters:
I20260925 22:02:07.344192 20910 leader_election.cc:288] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:02:07.344383 20916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:02:07.344470 20910 sys_catalog.cc:564] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:02:07.344595 20916 raft_consensus.cc:686] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [term 1 LEADER]: Becoming Leader. State: Replica: e87689ec013e4e3590c46d848cca1dcf, State: Running, Role: LEADER
I20260925 22:02:07.344741 20916 consensus_queue.cc:237] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87689ec013e4e3590c46d848cca1dcf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33159 } }
I20260925 22:02:07.345110 20916 sys_catalog.cc:455] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e87689ec013e4e3590c46d848cca1dcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87689ec013e4e3590c46d848cca1dcf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33159 } } }
I20260925 22:02:07.345254 20916 sys_catalog.cc:458] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [sys.catalog]: This master's current role is: LEADER
I20260925 22:02:07.345420 20916 sys_catalog.cc:455] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e87689ec013e4e3590c46d848cca1dcf. Latest consensus state: current_term: 1 leader_uuid: "e87689ec013e4e3590c46d848cca1dcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87689ec013e4e3590c46d848cca1dcf" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 33159 } } }
I20260925 22:02:07.345521 20916 sys_catalog.cc:458] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf [sys.catalog]: This master's current role is: LEADER
I20260925 22:02:07.345886 20910 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 22:02:07.348445 20924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:02:07.350106 20924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 22:02:07 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:07.357481 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:07.343252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.9.112.126:33159)
0925 22:02:07.343442 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:07.343452 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:07.343534 (+ 82us) server_negotiation.cc:407] Connection header received
0925 22:02:07.343677 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:07.343686 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:07.343754 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:07.343905 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:07.344648 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:07.346056 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:07.347667 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:07.349124 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:07.349805 (+ 681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:07.353068 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:07.353074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:07.353074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:07.353089 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:07.355169 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:07.355631 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:07.355635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:07.355635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:07.355758 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:07.355960 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:07.355960 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:07.355961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:07.356319 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:07.356970 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:07.357044 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 22:02:07.357179 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260925 22:02:07.360176 20924 catalog_manager.cc:1347] Generated new cluster ID: 2e952ca2392c46179270046ad6f9ccef
I20260925 22:02:07.360245 20924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:02:07.382441 20924 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:02:07.383280 20924 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:02:07.388525 20924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf: Generated new TSK 0
I20260925 22:02:07.388814 20924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:02:08.117126 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:35737/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33159
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:08.209848 20955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:08.210083 20955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:08.210117 20955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:08.211428 20955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:08.211484 20955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:08.211524 20955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:02:08.213208 20955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.112.84:35737/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:08.213480 20955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:08.213709 20955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:08.219213 20963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:08.219213 20961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:08.219304 20960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:08.219537 20955 server_base.cc:1034] running on GCE node
I20260925 22:02:08.219764 20955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:08.220044 20955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:08.221240 20955 hybrid_clock.cc:648] HybridClock initialized: now 1790373728221215 us; error 41 us; skew 500 ppm
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:08.224545 20955 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:02:08.225137 20955 webserver.cc:460] Webserver started at http://127.9.112.65:46123/ using document root <none> and password file <none>
I20260925 22:02:08.225405 20955 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:08.225466 20955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:08.225598 20955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:08.239648 20955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "628ba8b9e848428bbeff31304a52f6b7"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "8630c3e7c124a23f870907c50c952877"
server_key_iv: "2058a725b7d479dba9c953d0d5ea3beb"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.240083 20955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "628ba8b9e848428bbeff31304a52f6b7"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "8630c3e7c124a23f870907c50c952877"
server_key_iv: "2058a725b7d479dba9c953d0d5ea3beb"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.242079 20955 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 22:02:08.261981 20970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.264546 20955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 22:02:08.264696 20955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "628ba8b9e848428bbeff31304a52f6b7"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "8630c3e7c124a23f870907c50c952877"
server_key_iv: "2058a725b7d479dba9c953d0d5ea3beb"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.264809 20955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:08.275575 20955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:08.277599 20955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:08.277746 20955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:08.277982 20955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:08.278285 20955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:08.278317 20955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.278347 20955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:08.278363 20955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.284521 20955 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:45793
I20260925 22:02:08.284642 21083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:45793 every 8 connection(s)
I20260925 22:02:08.284977 20955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:08.293310 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 20955
I20260925 22:02:08.293426 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:02:08.293622 9665 external_mini_cluster.cc:1428] Setting key ac1ae9cdeb0e8815ad232def26bf025d
I20260925 22:02:08.295300 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.285695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33159 (local address 127.9.112.65:36871)
0925 22:02:08.285877 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:08.285928 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:08.286452 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:08.286702 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:08.286706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:08.286914 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:08.287112 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.287116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.287792 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.287793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.288712 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.288715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.289295 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.289296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.289439 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.290016 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:08.290025 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:08.291507 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:08.293463 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:08.293473 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:08.293476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:08.293813 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:08.294055 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:08.294056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:08.294056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:08.294215 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:08.294529 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:08.294537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:08.294739 (+ 202us) client_negotiation.cc:769] Sending connection context
0925 22:02:08.294866 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 22:02:08.295051 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260925 22:02:08.295775 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.285747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:36871 (local address 127.9.112.126:33159)
0925 22:02:08.285903 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.285906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.286016 (+ 110us) server_negotiation.cc:407] Connection header received
0925 22:02:08.286551 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.286553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.286566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.286631 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:08.287204 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.287705 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.288813 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.289135 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.289227 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.291621 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:08.291623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:08.291623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:08.291633 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:08.293318 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.293907 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.293909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.293909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.293979 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.294289 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.294290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.294290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.294447 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:08.294520 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.295588 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.295681 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260925 22:02:08.299218 21084 heartbeater.cc:347] Connected to a master server at 127.9.112.126:33159
I20260925 22:02:08.299346 21084 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:08.299566 21084 heartbeater.cc:511] Master 127.9.112.126:33159 requested a full tablet report, sending...
I20260925 22:02:08.300449 20874 ts_manager.cc:194] Registered new tserver with Master: 628ba8b9e848428bbeff31304a52f6b7 (127.9.112.65:45793)
I20260925 22:02:08.301403 20874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:36871
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:02:08.323820 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:35737/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33159
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:08.437469 21092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:08.437790 21092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:08.437835 21092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:08.439718 21092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:08.439781 21092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:08.439827 21092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:02:08.442330 21092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.112.84:35737/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:08.442623 21092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:08.442977 21092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:08.449576 21102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:08.449570 21099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:08.450002 21100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:08.450587 21092 server_base.cc:1034] running on GCE node
I20260925 22:02:08.450860 21092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:08.451166 21092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:08.452368 21092 hybrid_clock.cc:648] HybridClock initialized: now 1790373728452344 us; error 40 us; skew 500 ppm
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:08.455690 21092 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:02:08.456331 21092 webserver.cc:460] Webserver started at http://127.9.112.66:33029/ using document root <none> and password file <none>
I20260925 22:02:08.456643 21092 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:08.456715 21092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:08.456910 21092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:08.477002 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "c8d0d0c012074d6499acbb24137395c0"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "b8393379cd840736f3ad2d7aec829cbb"
server_key_iv: "9b65c8e8b13106267424f6637568ea03"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.477602 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "c8d0d0c012074d6499acbb24137395c0"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "b8393379cd840736f3ad2d7aec829cbb"
server_key_iv: "9b65c8e8b13106267424f6637568ea03"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.479483 21092 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:08.494859 21109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.495209 21092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:02:08.495308 21092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "c8d0d0c012074d6499acbb24137395c0"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "b8393379cd840736f3ad2d7aec829cbb"
server_key_iv: "9b65c8e8b13106267424f6637568ea03"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.495407 21092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:08.522395 21092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:08.524952 21092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:08.525214 21092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:08.525525 21092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:08.525944 21092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:08.526034 21092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.526113 21092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:08.526180 21092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.543190 21092 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:34405
I20260925 22:02:08.543582 21092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:02:08.544503 21222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:34405 every 8 connection(s)
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:08.553047 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 21092
I20260925 22:02:08.553148 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:02:08.553277 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.544414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33159 (local address 127.9.112.66:55293)
0925 22:02:08.544821 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:08.544861 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:08.545367 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:08.545612 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:08.545616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:08.545842 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:08.546080 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.546084 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.546763 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.546764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.547620 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.547623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.548181 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.548182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.548323 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.548886 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:08.548894 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:08.550114 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:08.551623 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:08.551631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:08.551634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:08.551875 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:08.552089 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:08.552090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:08.552091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:08.552223 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:08.552569 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:08.552575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:08.552752 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 22:02:08.552850 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 22:02:08.553020 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":90,"threads_started":1}
I20260925 22:02:08.553331 9665 external_mini_cluster.cc:1428] Setting key 92131953e7ae2d1cd9870750c6a8b691
I20260925 22:02:08.553980 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.544729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:55293 (local address 127.9.112.126:33159)
0925 22:02:08.544795 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.544797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.544964 (+ 167us) server_negotiation.cc:407] Connection header received
0925 22:02:08.545471 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.545473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.545488 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.545539 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:08.546163 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.546662 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.547717 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.548043 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.548105 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.550202 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:08.550204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:08.550204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:08.550212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:08.551544 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.551960 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.551962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.551962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.552021 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.552292 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.552293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.552294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.552474 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:08.552519 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.553787 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.553890 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:02:08.554446 21223 heartbeater.cc:347] Connected to a master server at 127.9.112.126:33159
I20260925 22:02:08.554548 21223 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:08.554730 21223 heartbeater.cc:511] Master 127.9.112.126:33159 requested a full tablet report, sending...
I20260925 22:02:08.555104 20874 ts_manager.cc:194] Registered new tserver with Master: c8d0d0c012074d6499acbb24137395c0 (127.9.112.66:34405)
I20260925 22:02:08.555553 20874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:55293
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:02:08.588310 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:35737/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33159
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:08.716985 21229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:08.717393 21229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:08.717480 21229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:08.719378 21229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:08.719509 21229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:08.719592 21229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:02:08.722081 21229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.112.84:35737/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:33159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:08.722606 21229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:08.722939 21229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:08.730127 21235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:08.730144 21234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:08.730484 21237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:08.730669 21229 server_base.cc:1034] running on GCE node
I20260925 22:02:08.731062 21229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:08.731350 21229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:08.732496 21229 hybrid_clock.cc:648] HybridClock initialized: now 1790373728732478 us; error 38 us; skew 500 ppm
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:08.735911 21229 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:02:08.736629 21229 webserver.cc:460] Webserver started at http://127.9.112.67:33083/ using document root <none> and password file <none>
I20260925 22:02:08.736918 21229 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:08.737021 21229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:08.737201 21229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:08.759173 21229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "41eb409941234d5f8955710c60af8d27"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "f1342ec0737e9be79602c1cec3b03143"
server_key_iv: "9e92bdd9abdf8246c5b90b1627b73de8"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.759860 21229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "41eb409941234d5f8955710c60af8d27"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "f1342ec0737e9be79602c1cec3b03143"
server_key_iv: "9e92bdd9abdf8246c5b90b1627b73de8"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.762782 21229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260925 22:02:08.788544 21244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.788915 21229 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260925 22:02:08.789011 21229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "41eb409941234d5f8955710c60af8d27"
format_stamp: "Formatted at 2026-09-25 22:02:08 on dist-test-slave-794d"
server_key: "f1342ec0737e9be79602c1cec3b03143"
server_key_iv: "9e92bdd9abdf8246c5b90b1627b73de8"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:08.789108 21229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:08.804584 21229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:08.807543 21229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:08.807798 21229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:08.808145 21229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:08.808607 21229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:08.808717 21229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.808804 21229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:08.808864 21229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:08.816883 21229 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:40501
I20260925 22:02:08.817278 21229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:02:08.822417 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 21229
I20260925 22:02:08.822517 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:02:08.822726 9665 external_mini_cluster.cc:1428] Setting key db1e04ea5954b1cdbc28ebe4e99a1b69
Sep 25 22:02:08 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:08.824517 21357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:40501 every 8 connection(s)
I20260925 22:02:08.826305 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.817957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:57951 (local address 127.9.112.126:33159)
0925 22:02:08.818192 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.818195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.818276 (+ 81us) server_negotiation.cc:407] Connection header received
0925 22:02:08.818801 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.818803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.818816 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.818866 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:08.819454 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.819961 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.820991 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.821201 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.821258 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.823552 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:08.823553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:08.823554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:08.823561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:08.825030 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.825418 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.825420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.825420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.825477 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.825691 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.825692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.825692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.825807 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:08.825880 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.826136 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.826214 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260925 22:02:08.826498 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.817964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:33159 (local address 127.9.112.67:57951)
0925 22:02:08.818127 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:08.818180 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:08.818696 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:08.818940 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:08.818943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:08.819141 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:08.819367 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.819372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.820052 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.820052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.820905 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.820908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.821272 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.821273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.821402 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.821906 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:08.821914 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:08.823457 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:08.825104 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:08.825112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:08.825114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:08.825349 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:08.825531 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:08.825531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:08.825532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:08.825631 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:08.825863 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:08.825869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:08.826012 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 22:02:08.826103 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:02:08.826267 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260925 22:02:08.827067 21358 heartbeater.cc:347] Connected to a master server at 127.9.112.126:33159
I20260925 22:02:08.827180 21358 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:08.827361 21358 heartbeater.cc:511] Master 127.9.112.126:33159 requested a full tablet report, sending...
I20260925 22:02:08.827726 20874 ts_manager.cc:194] Registered new tserver with Master: 41eb409941234d5f8955710c60af8d27 (127.9.112.67:40501)
I20260925 22:02:08.828114 20874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:57951
I20260925 22:02:08.834826 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:02:08.847074 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.836198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.9.112.126:33159)
0925 22:02:08.836250 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.836252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.840482 (+ 4230us) server_negotiation.cc:407] Connection header received
0925 22:02:08.840564 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.840565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.840578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.840627 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:08.841264 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.841803 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.842575 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.842769 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.842826 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.844549 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:08.844551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:08.844551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:08.844559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:08.846004 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.846334 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.846335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.846336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.846391 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:08.846566 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:08.846566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:08.846567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:08.846683 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:08.846787 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.846896 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.846979 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:02:08.866899 20874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:02:08.869264 20874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:02:08.883229 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.872699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:34405 (local address 127.0.0.1:39244)
0925 22:02:08.873907 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:08.873947 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:08.874248 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:08.874985 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:08.874988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:08.874990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:02:08.875058 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.875060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.879166 (+ 4106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.879167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.880286 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.880289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.882906 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.882907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.882994 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.883007 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:02:08.883069 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 22:02:08.883126 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"thread_start_us":68,"threads_started":1}
I20260925 22:02:08.883785 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.873330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:45793 (local address 127.0.0.1:57626)
0925 22:02:08.874460 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:08.874466 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:08.874497 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:08.875119 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:08.875119 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:08.875120 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:02:08.875143 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.875144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.876620 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.876621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.880906 (+ 4285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.880907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.883560 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.883562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.883625 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.883629 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:02:08.883678 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:02:08.883719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"thread_start_us":54,"threads_started":1}
I20260925 22:02:08.884737 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.873529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.9.112.65:45793)
0925 22:02:08.874226 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.874231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.874539 (+ 308us) server_negotiation.cc:407] Connection header received
0925 22:02:08.874576 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.874579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.874595 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.874654 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:02:08.875214 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.876534 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.880979 (+ 4445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.881780 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.881849 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.881907 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.884598 (+ 2691us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.884657 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":48,"threads_started":1}
I20260925 22:02:08.885546 21018 tablet_service.cc:1467] Processing CreateTablet for tablet 951668dee53d4e47aace23ff7f5a6db7 (DEFAULT_TABLE table=test-table [id=4039e5aadb8447d088ae13cc71cea453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:02:08.885913 21018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 951668dee53d4e47aace23ff7f5a6db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:02:08.890369 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.873112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.9.112.66:34405)
0925 22:02:08.874784 (+ 1672us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.874788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.874797 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:02:08.874830 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.874832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.874845 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.874906 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:02:08.878015 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.879062 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.881968 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.882736 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.882803 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.882873 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.890191 (+ 7318us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.890274 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1636,"thread_start_us":68,"threads_started":1}
I20260925 22:02:08.891242 21157 tablet_service.cc:1467] Processing CreateTablet for tablet 951668dee53d4e47aace23ff7f5a6db7 (DEFAULT_TABLE table=test-table [id=4039e5aadb8447d088ae13cc71cea453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:02:08.891780 21157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 951668dee53d4e47aace23ff7f5a6db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:02:08.893864 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.872917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40501 (local address 127.0.0.1:52662)
0925 22:02:08.874363 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:08.874372 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:08.874404 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:08.880727 (+ 6323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:08.880728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:08.880730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:02:08.880774 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.880776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.881993 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.881994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.883042 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:08.883044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.893553 (+ 10509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.893554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:08.893643 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.893648 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:02:08.893719 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 22:02:08.893775 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"thread_start_us":67,"threads_started":1}
I20260925 22:02:08.894191 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:08.873041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.9.112.67:40501)
0925 22:02:08.880454 (+ 7413us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:08.880460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:08.880471 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:02:08.880514 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:08.880516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:08.880535 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:08.880625 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:02:08.880848 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.881906 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.892430 (+ 10524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:08.893294 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:08.893409 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:08.893496 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:08.894065 (+ 569us) server_negotiation.cc:299] Negotiation successful
0925 22:02:08.894113 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7365,"thread_start_us":90,"threads_started":1}
I20260925 22:02:08.894910 21292 tablet_service.cc:1467] Processing CreateTablet for tablet 951668dee53d4e47aace23ff7f5a6db7 (DEFAULT_TABLE table=test-table [id=4039e5aadb8447d088ae13cc71cea453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:02:08.895308 21292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 951668dee53d4e47aace23ff7f5a6db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:02:08.896917 21377 tablet_bootstrap.cc:492] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: Bootstrap starting.
I20260925 22:02:08.897768 21377 tablet_bootstrap.cc:654] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: Neither blocks nor log segments found. Creating new log.
I20260925 22:02:08.898010 21378 tablet_bootstrap.cc:492] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Bootstrap starting.
I20260925 22:02:08.899322 21378 tablet_bootstrap.cc:654] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Neither blocks nor log segments found. Creating new log.
I20260925 22:02:08.899827 21377 log.cc:824] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: Log is configured to *not* fsync() on all Append() calls
I20260925 22:02:08.899991 21379 tablet_bootstrap.cc:492] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Bootstrap starting.
I20260925 22:02:08.900904 21378 log.cc:824] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:02:08.901098 21379 tablet_bootstrap.cc:654] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Neither blocks nor log segments found. Creating new log.
I20260925 22:02:08.901464 21379 log.cc:824] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Log is configured to *not* fsync() on all Append() calls
I20260925 22:02:08.901909 21377 tablet_bootstrap.cc:492] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: No bootstrap required, opened a new log
I20260925 22:02:08.902024 21377 ts_tablet_manager.cc:1397] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 22:02:08.904703 21377 raft_consensus.cc:348] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:08.905040 21377 raft_consensus.cc:374] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:02:08.905126 21377 raft_consensus.cc:729] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 628ba8b9e848428bbeff31304a52f6b7, State: Initialized, Role: FOLLOWER
I20260925 22:02:08.905337 21377 consensus_queue.cc:260] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:08.905709 21377 ts_tablet_manager.cc:1428] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 22:02:08.905860 21084 heartbeater.cc:503] Master 127.9.112.126:33159 was elected leader, sending a full tablet report...
I20260925 22:02:08.907572 21378 tablet_bootstrap.cc:492] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: No bootstrap required, opened a new log
I20260925 22:02:08.907711 21378 ts_tablet_manager.cc:1397] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260925 22:02:08.910392 21378 raft_consensus.cc:348] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:08.910676 21378 raft_consensus.cc:374] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:02:08.910758 21378 raft_consensus.cc:729] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d0d0c012074d6499acbb24137395c0, State: Initialized, Role: FOLLOWER
I20260925 22:02:08.910945 21378 consensus_queue.cc:260] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:08.911299 21378 ts_tablet_manager.cc:1428] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 22:02:08.911458 21223 heartbeater.cc:503] Master 127.9.112.126:33159 was elected leader, sending a full tablet report...
I20260925 22:02:08.913923 21379 tablet_bootstrap.cc:492] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: No bootstrap required, opened a new log
I20260925 22:02:08.914003 21379 ts_tablet_manager.cc:1397] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20260925 22:02:08.916550 21379 raft_consensus.cc:348] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:08.916860 21379 raft_consensus.cc:374] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:02:08.916952 21379 raft_consensus.cc:729] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41eb409941234d5f8955710c60af8d27, State: Initialized, Role: FOLLOWER
I20260925 22:02:08.917163 21379 consensus_queue.cc:260] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:08.917621 21379 ts_tablet_manager.cc:1428] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260925 22:02:08.917816 21358 heartbeater.cc:503] Master 127.9.112.126:33159 was elected leader, sending a full tablet report...
I20260925 22:02:09.008839 21383 raft_consensus.cc:482] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:02:09.008987 21383 raft_consensus.cc:504] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:09.009370 21383 leader_election.cc:288] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8d0d0c012074d6499acbb24137395c0 (127.9.112.66:34405), 41eb409941234d5f8955710c60af8d27 (127.9.112.67:40501)
I20260925 22:02:09.020176 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:09.010226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:55187 (local address 127.9.112.67:40501)
0925 22:02:09.010297 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:09.010300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:09.014007 (+ 3707us) server_negotiation.cc:407] Connection header received
0925 22:02:09.014118 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:09.014120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:09.014129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:09.014156 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:02:09.014367 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.015024 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.019202 (+ 4178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.019775 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.019828 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:09.019837 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:09.020060 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 22:02:09.020098 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 22:02:09.020457 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:09.009885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40501 (local address 127.9.112.65:55187)
0925 22:02:09.013878 (+ 3993us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:09.013897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:09.014041 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:09.014214 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:09.014215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:09.014216 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:02:09.014301 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:09.014303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.015158 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.015160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:09.016281 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:09.016286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.019906 (+ 3620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.019907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:09.019990 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:09.019993 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:02:09.020290 (+ 297us) client_negotiation.cc:240] Negotiation successful
0925 22:02:09.020365 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3968,"thread_start_us":79,"threads_started":1}
I20260925 22:02:09.021474 21312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951668dee53d4e47aace23ff7f5a6db7" candidate_uuid: "628ba8b9e848428bbeff31304a52f6b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41eb409941234d5f8955710c60af8d27" is_pre_election: true
I20260925 22:02:09.021656 21312 raft_consensus.cc:2454] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 628ba8b9e848428bbeff31304a52f6b7 in term 0.
I20260925 22:02:09.022102 20971 leader_election.cc:302] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41eb409941234d5f8955710c60af8d27, 628ba8b9e848428bbeff31304a52f6b7; no voters:
I20260925 22:02:09.022276 21383 raft_consensus.cc:2781] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:02:09.022327 21383 raft_consensus.cc:482] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:02:09.022347 21383 raft_consensus.cc:3037] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:02:09.023253 21383 raft_consensus.cc:504] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:09.023411 21383 leader_election.cc:288] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [CANDIDATE]: Term 1 election: Requested vote from peers c8d0d0c012074d6499acbb24137395c0 (127.9.112.66:34405), 41eb409941234d5f8955710c60af8d27 (127.9.112.67:40501)
I20260925 22:02:09.023859 21312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951668dee53d4e47aace23ff7f5a6db7" candidate_uuid: "628ba8b9e848428bbeff31304a52f6b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41eb409941234d5f8955710c60af8d27"
I20260925 22:02:09.023943 21312 raft_consensus.cc:3037] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:02:09.030035 21312 raft_consensus.cc:2454] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 628ba8b9e848428bbeff31304a52f6b7 in term 1.
I20260925 22:02:09.030405 20971 leader_election.cc:302] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41eb409941234d5f8955710c60af8d27, 628ba8b9e848428bbeff31304a52f6b7; no voters:
I20260925 22:02:09.030617 21383 raft_consensus.cc:2781] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:02:09.030793 21383 raft_consensus.cc:686] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [term 1 LEADER]: Becoming Leader. State: Replica: 628ba8b9e848428bbeff31304a52f6b7, State: Running, Role: LEADER
I20260925 22:02:09.030902 21383 consensus_queue.cc:237] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } }
I20260925 22:02:09.031935 20873 catalog_manager.cc:5495] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 628ba8b9e848428bbeff31304a52f6b7 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "628ba8b9e848428bbeff31304a52f6b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "628ba8b9e848428bbeff31304a52f6b7" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 45793 } health_report { overall_health: HEALTHY } } }
I20260925 22:02:09.036403 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:09.010181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:44141 (local address 127.9.112.66:34405)
0925 22:02:09.010266 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:09.010269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:09.010446 (+ 177us) server_negotiation.cc:407] Connection header received
0925 22:02:09.010538 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:09.010539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:09.010550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:09.010576 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:02:09.010798 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.011491 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.035235 (+ 23744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.035874 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.035931 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:09.035942 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:09.036241 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 22:02:09.036286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 22:02:09.036908 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:09.009648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:34405 (local address 127.9.112.65:44141)
0925 22:02:09.010381 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:09.010392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:09.010473 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:09.010638 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:09.010639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:09.010641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:02:09.010725 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:09.010727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.011614 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.011614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:09.035087 (+ 23473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:09.035093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.036060 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.036061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:09.036148 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:09.036153 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:02:09.036732 (+ 579us) client_negotiation.cc:240] Negotiation successful
0925 22:02:09.036793 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":101,"threads_started":1}
W20260925 22:02:09.041884 21085 tablet.cc:2369] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:02:09.044759 21175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951668dee53d4e47aace23ff7f5a6db7" candidate_uuid: "628ba8b9e848428bbeff31304a52f6b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8d0d0c012074d6499acbb24137395c0" is_pre_election: true
I20260925 22:02:09.044982 21175 raft_consensus.cc:2454] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 628ba8b9e848428bbeff31304a52f6b7 in term 0.
W20260925 22:02:09.045050 21224 tablet.cc:2369] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:02:09.045118 21177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "951668dee53d4e47aace23ff7f5a6db7" candidate_uuid: "628ba8b9e848428bbeff31304a52f6b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8d0d0c012074d6499acbb24137395c0"
I20260925 22:02:09.045181 21177 raft_consensus.cc:3037] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:02:09.046105 21177 raft_consensus.cc:2454] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 628ba8b9e848428bbeff31304a52f6b7 in term 1.
Sep 25 22:02:09 dist-test-slave-794d krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:09.069295 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:09.063316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.9.112.65:45793)
0925 22:02:09.063372 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:09.063374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:09.063384 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:02:09.063428 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:09.063429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:09.063457 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:09.063532 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:02:09.063737 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.064456 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.065882 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:09.066159 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:09.066225 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:09.068678 (+ 2453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:02:09.068832 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:02:09.068913 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:09.069156 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 22:02:09.069186 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 22:02:09.074954 21359 tablet.cc:2369] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:02:09.075740 21312 raft_consensus.cc:1261] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 1 FOLLOWER]: Refusing update from remote peer 628ba8b9e848428bbeff31304a52f6b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:02:09.075911 21177 raft_consensus.cc:1261] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 1 FOLLOWER]: Refusing update from remote peer 628ba8b9e848428bbeff31304a52f6b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:02:09.076241 21383 consensus_queue.cc:1035] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41eb409941234d5f8955710c60af8d27" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:02:09.076458 21383 consensus_queue.cc:1035] T 951668dee53d4e47aace23ff7f5a6db7 P 628ba8b9e848428bbeff31304a52f6b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8d0d0c012074d6499acbb24137395c0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:02:09.081666 21391 mvcc.cc:205] Tried to move back new op lower bound from 7333370794291085312 to 7333370794111291392. Current Snapshot: MvccSnapshot[applied={T|T < 7333370794291085312}]
I20260925 22:02:09.082113 21392 mvcc.cc:205] Tried to move back new op lower bound from 7333370794291085312 to 7333370794111291392. Current Snapshot: MvccSnapshot[applied={T|T < 7333370794291085312}]
I20260925 22:02:09.085412 21394 mvcc.cc:205] Tried to move back new op lower bound from 7333370794291085312 to 7333370794111291392. Current Snapshot: MvccSnapshot[applied={T|T < 7333370794291085312}]
I20260925 22:02:09.086382 20873 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:02:09.086493 20873 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:02:09.087253 20873 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e87689ec013e4e3590c46d848cca1dcf: Sending DeleteTablet for 3 replicas of tablet 951668dee53d4e47aace23ff7f5a6db7
I20260925 22:02:09.087909 21157 tablet_service.cc:1514] Processing DeleteTablet for tablet 951668dee53d4e47aace23ff7f5a6db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:02:09 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:39244
I20260925 22:02:09.088145 21292 tablet_service.cc:1514] Processing DeleteTablet for tablet 951668dee53d4e47aace23ff7f5a6db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:02:09 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:52662
I20260925 22:02:09.088378 21018 tablet_service.cc:1514] Processing DeleteTablet for tablet 951668dee53d4e47aace23ff7f5a6db7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:02:09 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:57626
I20260925 22:02:09.088968 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 20955
I20260925 22:02:09.089998 21400 tablet_replica.cc:330] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: stopping tablet replica
I20260925 22:02:09.090126 21400 raft_consensus.cc:2229] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:02:09.090245 21400 raft_consensus.cc:2258] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:02:09.090520 21401 tablet_replica.cc:330] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: stopping tablet replica
I20260925 22:02:09.090621 21401 raft_consensus.cc:2229] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:02:09.090718 21401 raft_consensus.cc:2258] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:02:09.091252 21401 ts_tablet_manager.cc:1905] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:02:09.093000 21400 ts_tablet_manager.cc:1905] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:02:09.096879 21400 ts_tablet_manager.cc:1918] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:02:09.097520 21401 ts_tablet_manager.cc:1918] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:02:09.097666 21400 log.cc:1196] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/951668dee53d4e47aace23ff7f5a6db7
I20260925 22:02:09.097987 21400 ts_tablet_manager.cc:1939] T 951668dee53d4e47aace23ff7f5a6db7 P c8d0d0c012074d6499acbb24137395c0: Deleting consensus metadata
I20260925 22:02:09.099072 21401 log.cc:1196] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/951668dee53d4e47aace23ff7f5a6db7
I20260925 22:02:09.099125 20859 catalog_manager.cc:4841] TS c8d0d0c012074d6499acbb24137395c0 (127.9.112.66:34405): tablet 951668dee53d4e47aace23ff7f5a6db7 (table test-table [id=4039e5aadb8447d088ae13cc71cea453]) successfully deleted
I20260925 22:02:09.100931 21401 ts_tablet_manager.cc:1939] T 951668dee53d4e47aace23ff7f5a6db7 P 41eb409941234d5f8955710c60af8d27: Deleting consensus metadata
I20260925 22:02:09.102780 20858 catalog_manager.cc:4841] TS 41eb409941234d5f8955710c60af8d27 (127.9.112.67:40501): tablet 951668dee53d4e47aace23ff7f5a6db7 (table test-table [id=4039e5aadb8447d088ae13cc71cea453]) successfully deleted
W20260925 22:02:09.103247 20860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:45793 (error 108)
I20260925 22:02:09.103605 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 21092
I20260925 22:02:09.104938 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:09.104805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:45793 (local address 127.0.0.1:57638)
0925 22:02:09.104858 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:09.104900 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:45793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:02:09.105054 20860 catalog_manager.cc:4584] TS 628ba8b9e848428bbeff31304a52f6b7 (127.9.112.65:45793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 951668dee53d4e47aace23ff7f5a6db7: Network error: Client connection negotiation failed: client connection to 127.9.112.65:45793: connect: Connection refused (error 111)
I20260925 22:02:09.111749 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 21229
I20260925 22:02:09.119961 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 20842
I20260925 22:02:09.128998 9665 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 22:02:09.473615 9665 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 22:02:09.473745 9665 mini_ranger.cc:66] Stopping Ranger...
I20260925 22:02:09.808437 9665 mini_ranger.cc:68] Stopped Ranger
2026-09-25 22:02:09.808 UTC [20428] LOG: received smart shutdown request
2026-09-25 22:02:09.812 UTC [20428] LOG: background worker "logical replication launcher" (PID 20437) exited with exit code 1
2026-09-25 22:02:09.813 UTC [20432] LOG: shutting down
2026-09-25 22:02:10.018 UTC [20428] LOG: database system is shut down
2026-09-25T22:02:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47592 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:02:10 dist-test-slave-794d krb5kdc[21412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:02:10 dist-test-slave-794d krb5kdc[21412](info): set up 2 sockets
Sep 25 22:02:10 dist-test-slave-794d krb5kdc[21412](info): commencing operation
krb5kdc: starting...
W20260925 22:02:11.068157 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:02:11 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:02:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/rangeradmin_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/rangeradmin_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/rangerlookup_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/rangerlookup_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
I20260925 22:02:11.234820 9665 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQ0dh6G/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/postgres -l logfile start
2026-09-25 22:02:14.029 UTC [21442] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 22:02:14.029 UTC [21442] LOG: listening on IPv4 address "127.9.112.84", port 40355
2026-09-25 22:02:14.034 UTC [21442] LOG: listening on Unix socket "/tmp/.s.PGSQL.40355"
2026-09-25 22:02:14.063 UTC [21445] LOG: database system was shut down at 2026-09-25 22:02:12 UTC
2026-09-25 22:02:14.068 UTC [21442] LOG: database system is ready to accept connections
I20260925 22:02:14.948607 9665 mini_postgres.cc:96] Postgres bound to 40355
2026-09-25 22:02:14.955 UTC [21453] FATAL: database "slave" does not exist
127.9.112.84:40355 - accepting connections
I20260925 22:02:14.956396 9665 mini_ranger.cc:161] Starting Ranger...
I20260925 22:02:14.972886 9665 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 22:02:15.567296 9665 mini_ranger.cc:87] Created ranger Postgres database
I20260925 22:02:15.567394 9665 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-admin
2026-09-25 22:02:15,858 [I] DB FLAVOR :POSTGRES
2026-09-25 22:02:15,859 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:02:15,859 [I] Checking connection..
2026-09-25 22:02:15,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 22:02:16,130 [I] Checking connection passed.
2026-09-25 22:02:16,130 [I] --------- Verifying version history table ---------
2026-09-25 22:02:16,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:02:16,387 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:02:16,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:02:16,652 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 22:02:16,652 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 22:02:16,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 22:02:16.909 UTC [21517] WARNING: there is no transaction in progress
2026-09-25 22:02:16,916 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 22:02:16,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 22:02:17,209 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 22:02:17,210 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:02:17,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 22:02:17,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 22:02:17,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-794d.c.gcp-upstream.internal','N') ;"
2026-09-25 22:02:18,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:02:18,282 [I] Table x_portal_user does not exist in database ranger
2026-09-25 22:02:18,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:02:18,567 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 22:02:18,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:02:18,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:02:19,116 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 22:02:19,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 22:02:20.270 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.294 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.319 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.344 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.367 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.389 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.461 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.469 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.487 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.499 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.512 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.531 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.546 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.554 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.564 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:20.573 UTC [21625] WARNING: there is no transaction in progress
2026-09-25 22:02:21,381 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 22:02:21,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 22:02:21,657 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 22:02:21,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 22:02:21,947 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 22:02:21,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 22:02:22,244 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 22:02:22,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 22:02:22,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 22:02:22,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-794d.c.gcp-upstream.internal';"
2026-09-25 22:02:23,041 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 22:02:23,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 22:02:23,299 [I] DB_PATCHES have already been applied
I20260925 22:02:23.306783 9665 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQ0dh6G/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 22:02:23.306874 9665 mini_ranger.cc:193] Using host: 127.9.112.84
I20260925 22:02:23.310763 9665 mini_ranger.cc:227] Ranger admin URL: http://127.9.112.84:47785
Sep 25, 2026 10:02:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:02:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:02:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 10:02:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 10:02:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 10:02:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 10:02:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 22:02:24.338903 9665 mini_ranger.cc:160] Time spent starting Ranger: real 9.383s user 0.002s sys 0.008s
Sep 25, 2026 10:02:45 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 10:02:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 10:02:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 10:02:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 10:02:46 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 22:02:47.405627 9665 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.112.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.112.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/rangerkms_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.112.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/rangerkms_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.112.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.112.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.112.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/HTTP_127.9.112.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 22:02:47 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373767, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 22:02:47.493438 9665 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 22:02:47.502542 9665 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 22:02:48.513618 9665 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 22:02:48.513715 9665 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms
I20260925 22:02:48.513734 9665 mini_ranger_kms.cc:227] Using postgres at 127.9.112.84:40355
2026-09-25 22:02:48,706 [I] DB FLAVOR :POSTGRES
2026-09-25 22:02:48,706 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 22:02:48,706 [I] Checking connection
2026-09-25 22:02:48,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 22:02:48,955 [I] connection success
2026-09-25 22:02:48,955 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 22:02:48,955 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 22:02:48,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 22:02:49,224 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 22:02:49,224 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 22:02:49,225 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 22:02:49,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.112.84:40355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 22:02:49,517 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 22:02:50.100855 9665 mini_ranger_kms.cc:310] Created kms service
I20260925 22:02:50.299893 9665 mini_ranger_kms.cc:326] Created kudu user
I20260925 22:02:50.348138 9665 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 22:02:50 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373767, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:50.797642 9665 mini_ranger_kms.cc:384] Added ranger policy
I20260925 22:02:50.797765 9665 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQ0dh6G/build/release/bin/postgresql.jar:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQ0dh6G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 22:02:50.797783 9665 mini_ranger_kms.cc:240] Using host: 127.9.112.84
I20260925 22:02:50.801059 9665 mini_ranger_kms.cc:276] Ranger KMS PID: 21806
I20260925 22:02:50.801144 9665 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.112.84:47149
Sep 25, 2026 10:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 10:02:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 10:02:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 10:02:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 22:02:52.891956 9665 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.399s user 0.001s sys 0.174s
I20260925 22:02:52.892052 9665 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 22:02:52.892098 9665 mini_ranger_kms.cc:401] 127.9.112.84:47149/kms/v1/keys
Sep 25 22:02:53 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790373773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.112.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 22:02:53 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.112.84@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
Sep 25 22:02:53 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.112.84@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
Sep 25 22:02:56 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:02:56.403409 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45271
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:45271
--ranger_config_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:56.495913 21856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:56.496182 21856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:56.496204 21856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:56.497471 21856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:02:56.497522 21856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:56.497535 21856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:56.497546 21856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:02:56.497557 21856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:02:56.499342 21856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.112.84:47149/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:45271
--ranger_config_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:56.499615 21856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:56.499863 21856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:56.505386 21861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:56.505436 21862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:56.505474 21856 server_base.cc:1034] running on GCE node
W20260925 22:02:56.505576 21864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:56.506138 21856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:56.506560 21856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:56.507752 21856 hybrid_clock.cc:648] HybridClock initialized: now 1790373776507717 us; error 45 us; skew 500 ppm
Sep 25 22:02:56 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373776, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:56.511123 21856 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:02:56.511642 21856 webserver.cc:460] Webserver started at http://127.9.112.126:45401/ using document root <none> and password file <none>
I20260925 22:02:56.511910 21856 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:56.511953 21856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:56.512082 21856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:56 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373776, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:56.578374 21856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "a9c68c780a424db99c09c9b6ce81b318"
format_stamp: "Formatted at 2026-09-25 22:02:56 on dist-test-slave-794d"
server_key: "bbc58e3fea52bf41371e3709304fb5af"
server_key_iv: "de567775c066f60c5eae5dda13cbb2ff"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:56.579097 21856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "a9c68c780a424db99c09c9b6ce81b318"
format_stamp: "Formatted at 2026-09-25 22:02:56 on dist-test-slave-794d"
server_key: "bbc58e3fea52bf41371e3709304fb5af"
server_key_iv: "de567775c066f60c5eae5dda13cbb2ff"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:56.581317 21856 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260925 22:02:56.598446 21871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:56.598726 21856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:56.598860 21856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "a9c68c780a424db99c09c9b6ce81b318"
format_stamp: "Formatted at 2026-09-25 22:02:56 on dist-test-slave-794d"
server_key: "bbc58e3fea52bf41371e3709304fb5af"
server_key_iv: "de567775c066f60c5eae5dda13cbb2ff"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:56.598995 21856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:56.614163 21856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:56.616403 21856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:56.616568 21856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:56.622361 21856 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:45271
I20260925 22:02:56.622430 21923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:45271 every 8 connection(s)
I20260925 22:02:56.622956 21856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:02:56.623761 21924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:02:56.626571 21924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Bootstrap starting.
I20260925 22:02:56.627260 21924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Neither blocks nor log segments found. Creating new log.
I20260925 22:02:56.627565 21924 log.cc:824] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Log is configured to *not* fsync() on all Append() calls
I20260925 22:02:56.628261 21924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: No bootstrap required, opened a new log
I20260925 22:02:56.630483 21924 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:56.630686 21924 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:02:56.630723 21924 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c68c780a424db99c09c9b6ce81b318, State: Initialized, Role: FOLLOWER
I20260925 22:02:56.630770 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 21856
I20260925 22:02:56.630868 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:02:56.630926 21924 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:56.631009 21924 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:02:56.631060 21924 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:02:56.631093 9665 external_mini_cluster.cc:1428] Setting key 91efa415c078956b1d341d231a659f85
I20260925 22:02:56.631114 21924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:02:56.631850 21924 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:56.631976 21924 leader_election.cc:302] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c68c780a424db99c09c9b6ce81b318; no voters:
I20260925 22:02:56.632169 21924 leader_election.cc:288] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:02:56.632220 21929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:02:56.632498 21929 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 LEADER]: Becoming Leader. State: Replica: a9c68c780a424db99c09c9b6ce81b318, State: Running, Role: LEADER
I20260925 22:02:56.632587 21924 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:02:56.632694 21929 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:56.633104 21929 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c68c780a424db99c09c9b6ce81b318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } } }
I20260925 22:02:56.633232 21929 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: This master's current role is: LEADER
I20260925 22:02:56.633409 21929 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c68c780a424db99c09c9b6ce81b318. Latest consensus state: current_term: 1 leader_uuid: "a9c68c780a424db99c09c9b6ce81b318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } } }
I20260925 22:02:56.633502 21929 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: This master's current role is: LEADER
I20260925 22:02:56.633643 21938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:02:56.634187 21938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:02:56.634442 21924 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 22:02:56.640360 21938 catalog_manager.cc:1347] Generated new cluster ID: 889b34325d0542f5951807badbd3df31
I20260925 22:02:56.640475 21938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:02:56 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:56.660398 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:56.631662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.9.112.126:45271)
0925 22:02:56.631876 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:56.631888 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:56.631927 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:02:56.632064 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:56.632072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:56.632144 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:56.633221 (+ 1077us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:56.633350 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:56.634852 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:56.636747 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:56.638357 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:56.641463 (+ 3106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:56.641560 (+ 97us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:56.641563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:56.641564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:56.641573 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:56.657828 (+ 16255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:56.658442 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:56.658448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:56.658448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:56.658582 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:56.658799 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:56.658799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:56.658800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:56.659195 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:56.659831 (+ 636us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:56.659922 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 22:02:56.660081 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260925 22:02:56.663721 21938 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:02:56.664678 21938 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:02:56.676659 21938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Generated new TSK 0
I20260925 22:02:56.676960 21938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:02:57.440858 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45271
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:57.578675 21969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:57.579026 21969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:57.579077 21969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:57.581097 21969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:57.581158 21969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:57.581202 21969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:02:57.583621 21969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.112.84:47149/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:57.583941 21969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:57.584331 21969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:57.590981 21977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:57.591071 21974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:57.591082 21975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:57.591485 21969 server_base.cc:1034] running on GCE node
I20260925 22:02:57.591745 21969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:57.592062 21969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:57.593245 21969 hybrid_clock.cc:648] HybridClock initialized: now 1790373777593226 us; error 35 us; skew 500 ppm
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:57.596588 21969 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:02:57.597134 21969 webserver.cc:460] Webserver started at http://127.9.112.65:43217/ using document root <none> and password file <none>
I20260925 22:02:57.597386 21969 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:57.597440 21969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:57.597572 21969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:57.617079 21969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "b60ee4544ea14742a4ee41967e4b5452"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "e1a476c38264be225ed78a19e59798d2"
server_key_iv: "0bfb250d6bbf5c27d83bdc2f9cc309da"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:57.617658 21969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "b60ee4544ea14742a4ee41967e4b5452"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "e1a476c38264be225ed78a19e59798d2"
server_key_iv: "0bfb250d6bbf5c27d83bdc2f9cc309da"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:57.620806 21969 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 22:02:57.639760 21986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:57.640111 21969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 22:02:57.640247 21969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "b60ee4544ea14742a4ee41967e4b5452"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "e1a476c38264be225ed78a19e59798d2"
server_key_iv: "0bfb250d6bbf5c27d83bdc2f9cc309da"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:57.640475 21969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:57.707989 21969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:57.710803 21969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:57.711068 21969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:57.711424 21969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:57.711925 21969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:57.712023 21969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:57.712144 21969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:57.712236 21969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:57.720288 21969 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:39209
I20260925 22:02:57.721771 21969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:02:57.725025 22099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:39209 every 8 connection(s)
I20260925 22:02:57.731161 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 21969
I20260925 22:02:57.731276 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:02:57.731508 9665 external_mini_cluster.cc:1428] Setting key cb8e5ce9a84e940874fda033cfbdb2f8
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:57.738346 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:57.723601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45271 (local address 127.9.112.65:43617)
0925 22:02:57.724115 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:57.724194 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:57.725254 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:57.725707 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:57.725713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:57.726022 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:57.726351 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:57.726358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.727621 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.727622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:57.729094 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:57.729098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.729731 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.729732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:57.729903 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:57.730578 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:57.730590 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:57.733563 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:57.736019 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:57.736032 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:57.736036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:57.736687 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:57.736951 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:57.736952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:57.736952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:57.737127 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:57.737479 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:57.737487 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:57.737694 (+ 207us) client_negotiation.cc:769] Sending connection context
0925 22:02:57.737805 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 22:02:57.738028 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":111,"threads_started":1}
I20260925 22:02:57.738900 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:57.723962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:43617 (local address 127.9.112.126:45271)
0925 22:02:57.724647 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:57.724650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:57.724663 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:02:57.725456 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:57.725458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:57.725477 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:57.725545 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:57.726484 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.727497 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.729201 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.729563 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.729633 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:57.733688 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:57.733690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:57.733691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:57.733700 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:57.735909 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:57.736792 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:57.736793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:57.736794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:57.736878 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:57.737202 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:57.737203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:57.737203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:57.737376 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:57.737449 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:57.738681 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0925 22:02:57.738791 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":59,"threads_started":1}
I20260925 22:02:57.739375 22100 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45271
I20260925 22:02:57.739512 22100 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:57.739761 22100 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:57.740495 21888 ts_manager.cc:194] Registered new tserver with Master: b60ee4544ea14742a4ee41967e4b5452 (127.9.112.65:39209)
I20260925 22:02:57.741425 21888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:43617
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:02:57.772076 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45271
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 22:02:57.875669 22108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:57.876111 22108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:57.876199 22108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:57.878153 22108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:57.878283 22108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:57.878371 22108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:02:57.880223 22108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.112.84:47149/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:57.880529 22108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:57.880822 22108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:57.887689 22113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:57.887750 22114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:57.887774 22116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:57.888024 22108 server_base.cc:1034] running on GCE node
I20260925 22:02:57.888249 22108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:57.888533 22108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:57.889703 22108 hybrid_clock.cc:648] HybridClock initialized: now 1790373777889657 us; error 41 us; skew 500 ppm
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:57.893128 22108 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:02:57.893710 22108 webserver.cc:460] Webserver started at http://127.9.112.66:36587/ using document root <none> and password file <none>
I20260925 22:02:57.894002 22108 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:57.894061 22108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:57.894192 22108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:57.918959 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "6b104c5166f5483a8e750ba29519fec9"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "162ece4a68c3e5dd705d263e2ff6821a"
server_key_iv: "8050386d63380dfafe33eefd7f287c81"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:57.919613 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "6b104c5166f5483a8e750ba29519fec9"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "162ece4a68c3e5dd705d263e2ff6821a"
server_key_iv: "8050386d63380dfafe33eefd7f287c81"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:57.921837 22108 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 22:02:57.944563 22124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:57.944869 22108 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260925 22:02:57.945009 22108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "6b104c5166f5483a8e750ba29519fec9"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "162ece4a68c3e5dd705d263e2ff6821a"
server_key_iv: "8050386d63380dfafe33eefd7f287c81"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:57.945134 22108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:57.967314 22108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:57.969637 22108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:57.969811 22108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:57.970095 22108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:57.970498 22108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:57.970543 22108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:57.970578 22108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:57.970598 22108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:57.978688 22108 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:37387
I20260925 22:02:57.979149 22108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:02:57.979831 22237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:37387 every 8 connection(s)
Sep 25 22:02:57 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:57.989192 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22108
I20260925 22:02:57.989316 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:02:57.989567 9665 external_mini_cluster.cc:1428] Setting key 3c04e46042e9cff75a770c1405dca830
I20260925 22:02:57.989871 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:57.980421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:48023 (local address 127.9.112.126:45271)
0925 22:02:57.980490 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:57.980493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:57.980510 (+ 17us) server_negotiation.cc:407] Connection header received
0925 22:02:57.981025 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:57.981027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:57.981043 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:57.981095 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:57.981748 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.982345 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.983892 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.984142 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.984208 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:57.986655 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:57.986658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:57.986659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:57.986669 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:57.988107 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:57.988615 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:57.988617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:57.988617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:57.988677 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:57.988937 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:57.988938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:57.988939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:57.989119 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:57.989187 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:57.989630 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 22:02:57.989752 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:02:57.990413 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:57.980025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45271 (local address 127.9.112.66:48023)
0925 22:02:57.980206 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:57.980255 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:57.980901 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:57.981180 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:57.981184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:57.981413 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:57.981648 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:57.981653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.982437 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.982438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:57.983713 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:57.983717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:57.984237 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:57.984238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:57.984461 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:57.985072 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:57.985082 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:57.986528 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:57.988195 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:57.988206 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:57.988210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:57.988541 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:57.988735 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:57.988736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:57.988737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:57.988871 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:57.989215 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:57.989224 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:57.989454 (+ 230us) client_negotiation.cc:769] Sending connection context
0925 22:02:57.989975 (+ 521us) client_negotiation.cc:240] Negotiation successful
0925 22:02:57.990185 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260925 22:02:57.991029 22238 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45271
I20260925 22:02:57.991145 22238 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:57.991365 22238 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:57.991758 21888 ts_manager.cc:194] Registered new tserver with Master: 6b104c5166f5483a8e750ba29519fec9 (127.9.112.66:37387)
I20260925 22:02:57.992244 21888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:48023
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:02:58.024616 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45271
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:58.153090 22243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:58.153354 22243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:58.153391 22243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:58.155100 22243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:58.155169 22243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:58.155210 22243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:02:58.157618 22243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.112.84:47149/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:58.157958 22243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:58.158284 22243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:58.166267 22251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.166606 22249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.166692 22248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:58.167935 22243 server_base.cc:1034] running on GCE node
I20260925 22:02:58.168215 22243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:58.168596 22243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:58.169799 22243 hybrid_clock.cc:648] HybridClock initialized: now 1790373778169763 us; error 51 us; skew 500 ppm
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:58.173702 22243 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:02:58.174388 22243 webserver.cc:460] Webserver started at http://127.9.112.67:41937/ using document root <none> and password file <none>
I20260925 22:02:58.174732 22243 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:58.174865 22243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:02:58.175076 22243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:58.195571 22243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "5a0f400626004a9cb5ab483d6f11212e"
format_stamp: "Formatted at 2026-09-25 22:02:58 on dist-test-slave-794d"
server_key: "0a1d16b61fa777067bc1813d654aff06"
server_key_iv: "df90b21480444793f6e17928f86e8bfa"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:58.197243 22243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "5a0f400626004a9cb5ab483d6f11212e"
format_stamp: "Formatted at 2026-09-25 22:02:58 on dist-test-slave-794d"
server_key: "0a1d16b61fa777067bc1813d654aff06"
server_key_iv: "df90b21480444793f6e17928f86e8bfa"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:58.199962 22243 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 22:02:58.220149 22258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.220443 22243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260925 22:02:58.220563 22243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "5a0f400626004a9cb5ab483d6f11212e"
format_stamp: "Formatted at 2026-09-25 22:02:58 on dist-test-slave-794d"
server_key: "0a1d16b61fa777067bc1813d654aff06"
server_key_iv: "df90b21480444793f6e17928f86e8bfa"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:58.220686 22243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:58.243333 22243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:58.246125 22243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:58.246523 22243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:58.246860 22243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:58.247407 22243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:58.247524 22243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.247608 22243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:58.247668 22243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.254874 22243 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:42527
I20260925 22:02:58.255409 22243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:02:58.256933 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22243
I20260925 22:02:58.257032 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:02:58.257253 9665 external_mini_cluster.cc:1428] Setting key 20373c9c358d5d2c51ebab174f60d52c
I20260925 22:02:58.256737 22371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:42527 every 8 connection(s)
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:58.267582 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:58.257453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:53061 (local address 127.9.112.126:45271)
0925 22:02:58.257508 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:58.257510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:58.258013 (+ 503us) server_negotiation.cc:407] Connection header received
0925 22:02:58.258801 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:58.258802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:58.258814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:58.258841 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:58.259831 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.260594 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.261912 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.262129 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.262163 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:58.264386 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:58.264388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:58.264388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:58.264396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:58.265852 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:58.266410 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:58.266411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:58.266412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:58.266476 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:58.266772 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:58.266773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:58.266774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:58.266918 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:58.266964 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:58.267377 (+ 413us) server_negotiation.cc:299] Negotiation successful
0925 22:02:58.267476 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:02:58.268131 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:58.256419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45271 (local address 127.9.112.67:53061)
0925 22:02:58.257870 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:58.257924 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:58.258681 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:58.259036 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:58.259040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:58.259285 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:58.259678 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:58.259688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.260687 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.260688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:58.261793 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:58.261796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.262255 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.262255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:58.262363 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:58.262883 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:58.262892 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:58.264270 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:58.265929 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:58.265939 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:58.265942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:58.266314 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:58.266549 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:58.266550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:58.266550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:58.266699 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:58.267024 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:58.267030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:58.267210 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 22:02:58.267699 (+ 489us) client_negotiation.cc:240] Negotiation successful
0925 22:02:58.267881 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"thread_start_us":89,"threads_started":1}
I20260925 22:02:58.268642 22372 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45271
I20260925 22:02:58.268795 22372 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.269071 22372 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:58.269495 21888 ts_manager.cc:194] Registered new tserver with Master: 5a0f400626004a9cb5ab483d6f11212e (127.9.112.67:42527)
I20260925 22:02:58.269876 21888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:53061
I20260925 22:02:58.280066 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:02:58.280804 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 21969
I20260925 22:02:58.286590 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22108
I20260925 22:02:58.290934 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22243
I20260925 22:02:58.295641 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 21856
I20260925 22:02:58.300869 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45271
--webserver_interface=127.9.112.126
--webserver_port=45401
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:45271
--ranger_config_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:02:58.389559 22375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:58.389799 22375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:58.389822 22375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:58.391070 22375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:02:58.391120 22375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:58.391134 22375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:58.391145 22375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:02:58.391155 22375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:02:58.392915 22375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:45271
--ranger_config_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:45271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=45401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:58.393160 22375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:58.393378 22375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:58.398072 22383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.398144 22381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.398144 22380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:58.398379 22375 server_base.cc:1034] running on GCE node
I20260925 22:02:58.398636 22375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:58.398910 22375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:58.400079 22375 hybrid_clock.cc:648] HybridClock initialized: now 1790373778400055 us; error 37 us; skew 500 ppm
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:58.402667 22375 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:02:58.403121 22375 webserver.cc:460] Webserver started at http://127.9.112.126:45401/ using document root <none> and password file <none>
I20260925 22:02:58.403322 22375 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:58.403369 22375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:58.457407 22375 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:58.458261 22391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.458459 22375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:58.458585 22375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "a9c68c780a424db99c09c9b6ce81b318"
format_stamp: "Formatted at 2026-09-25 22:02:56 on dist-test-slave-794d"
server_key: "bbc58e3fea52bf41371e3709304fb5af"
server_key_iv: "de567775c066f60c5eae5dda13cbb2ff"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:58.459117 22375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:58.482506 22375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:58.485051 22375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:58.485226 22375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:58.490403 22375 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:45271
I20260925 22:02:58.490435 22443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:45271 every 8 connection(s)
I20260925 22:02:58.490944 22375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:02:58.492587 22444 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:02:58.493261 22444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Bootstrap starting.
I20260925 22:02:58.497814 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22375
I20260925 22:02:58.498175 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:39209
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:45271
--webserver_port=43217
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:02:58.502067 22444 log.cc:824] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Log is configured to *not* fsync() on all Append() calls
I20260925 22:02:58.503772 22444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:02:58.504084 22444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Bootstrap complete.
I20260925 22:02:58.506388 22444 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:58.506640 22444 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c68c780a424db99c09c9b6ce81b318, State: Initialized, Role: FOLLOWER
I20260925 22:02:58.506853 22444 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:58.506923 22444 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:02:58.506989 22444 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:02:58.507056 22444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:02:58.508389 22444 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:58.508534 22444 leader_election.cc:302] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c68c780a424db99c09c9b6ce81b318; no voters:
I20260925 22:02:58.508754 22444 leader_election.cc:288] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:02:58.508810 22448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:02:58.509027 22448 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [term 2 LEADER]: Becoming Leader. State: Replica: a9c68c780a424db99c09c9b6ce81b318, State: Running, Role: LEADER
I20260925 22:02:58.509135 22444 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:02:58.509229 22448 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } }
I20260925 22:02:58.509590 22448 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a9c68c780a424db99c09c9b6ce81b318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } } }
I20260925 22:02:58.509749 22448 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: This master's current role is: LEADER
I20260925 22:02:58.509918 22448 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c68c780a424db99c09c9b6ce81b318. Latest consensus state: current_term: 2 leader_uuid: "a9c68c780a424db99c09c9b6ce81b318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c68c780a424db99c09c9b6ce81b318" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 45271 } } }
I20260925 22:02:58.510028 22448 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318 [sys.catalog]: This master's current role is: LEADER
I20260925 22:02:58.510180 22457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:02:58.510648 22444 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 22:02:58.510771 22457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:02:58.510963 22457 catalog_manager.cc:1259] Loaded cluster ID: 889b34325d0542f5951807badbd3df31
I20260925 22:02:58.511041 22457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:02:58.516690 22457 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:02:58.517244 22457 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Loaded TSK: 0
I20260925 22:02:58.517601 22457 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:02:58.627074 22446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:58.627390 22446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:58.627430 22446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:58.629555 22446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:58.629652 22446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:58.629705 22446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:02:58.632757 22446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:39209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=43217
--tserver_master_addrs=127.9.112.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:58.633077 22446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:58.633378 22446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:58.640504 22481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:58.640880 22446 server_base.cc:1034] running on GCE node
W20260925 22:02:58.640527 22482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.641165 22484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:58.641569 22446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:58.641909 22446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:58.643098 22446 hybrid_clock.cc:648] HybridClock initialized: now 1790373778643066 us; error 46 us; skew 500 ppm
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:58.646509 22446 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:02:58.647177 22446 webserver.cc:460] Webserver started at http://127.9.112.65:43217/ using document root <none> and password file <none>
I20260925 22:02:58.647440 22446 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:58.647523 22446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:58.672879 22446 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:58.673936 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.674103 22446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:02:58.674181 22446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "b60ee4544ea14742a4ee41967e4b5452"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "e1a476c38264be225ed78a19e59798d2"
server_key_iv: "0bfb250d6bbf5c27d83bdc2f9cc309da"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:58.674639 22446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:58.697494 22446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:58.699751 22446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:58.699918 22446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:58.700150 22446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:58.700515 22446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:58.700551 22446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.700583 22446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:58.700606 22446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.707886 22446 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:39209
I20260925 22:02:58.707985 22605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:39209 every 8 connection(s)
I20260925 22:02:58.708451 22446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:02:58.714134 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22446
I20260925 22:02:58.714455 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:37387
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:45271
--webserver_port=36587
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:58.723858 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:58.709276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45271 (local address 127.9.112.65:60857)
0925 22:02:58.709829 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:58.709868 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:58.710434 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:58.711008 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:58.711013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:58.711281 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:58.711613 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:58.711622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.713040 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.713041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:58.714165 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:58.714169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.715219 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.715220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:58.715364 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:58.716049 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:58.716061 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:58.718412 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:58.721719 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:58.721732 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:58.721735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:58.722091 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:58.722416 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:58.722417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:58.722418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:58.722565 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:58.723116 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:58.723122 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:58.723287 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 22:02:58.723403 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 22:02:58.723589 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":82,"threads_started":1}
I20260925 22:02:58.724421 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:58.709384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:60857 (local address 127.9.112.126:45271)
0925 22:02:58.709722 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:58.709734 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:58.709982 (+ 248us) server_negotiation.cc:407] Connection header received
0925 22:02:58.710700 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:58.710707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:58.710781 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:58.710928 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:58.711791 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.712916 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.714278 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.714958 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.715084 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:58.718634 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:58.718641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:58.718642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:58.718660 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:58.721538 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:58.722205 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:58.722210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:58.722210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:58.722340 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:58.722637 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:58.722638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:58.722639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:58.723000 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:58.723084 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:58.724063 (+ 979us) server_negotiation.cc:299] Negotiation successful
0925 22:02:58.724202 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260925 22:02:58.725124 22606 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45271
I20260925 22:02:58.725241 22606 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.725451 22606 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
W20260925 22:02:58.726421 22606 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:45271 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 22:02:58.726501 22606 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.726575 22606 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:58.727038 22606 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.727123 22606 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
W20260925 22:02:58.727473 22606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:02:58.827533 22610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:58.827762 22610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:58.827788 22610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:58.829435 22610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:58.829505 22610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:58.829538 22610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:02:58.831686 22610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:37387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=36587
--tserver_master_addrs=127.9.112.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:58.831979 22610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:58.832264 22610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:02:58.838582 22616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.838792 22618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:58.838945 22615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:58.839488 22610 server_base.cc:1034] running on GCE node
I20260925 22:02:58.839814 22610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:58.840082 22610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:58.841269 22610 hybrid_clock.cc:648] HybridClock initialized: now 1790373778841236 us; error 47 us; skew 500 ppm
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:58.844075 22610 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:02:58.844609 22610 webserver.cc:460] Webserver started at http://127.9.112.66:36587/ using document root <none> and password file <none>
I20260925 22:02:58.844834 22610 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:58.844895 22610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:58.857291 22610 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:58.858384 22626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.858609 22610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:02:58.858701 22610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "6b104c5166f5483a8e750ba29519fec9"
format_stamp: "Formatted at 2026-09-25 22:02:57 on dist-test-slave-794d"
server_key: "162ece4a68c3e5dd705d263e2ff6821a"
server_key_iv: "8050386d63380dfafe33eefd7f287c81"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:58.859198 22610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:58.874409 22610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:58.876917 22610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:58.877085 22610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:58.877354 22610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:58.877735 22610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:58.877774 22610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.877796 22610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:58.877807 22610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:58.885084 22610 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:37387
I20260925 22:02:58.885257 22739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:37387 every 8 connection(s)
I20260925 22:02:58.885583 22610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:02:58.891440 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22610
I20260925 22:02:58.891741 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.112.84:47149/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:42527
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:45271
--webserver_port=41937
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:02:58 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:58.900707 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:58.886441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:60279 (local address 127.9.112.126:45271)
0925 22:02:58.886547 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:58.886549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:58.886727 (+ 178us) server_negotiation.cc:407] Connection header received
0925 22:02:58.887275 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:58.887276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:58.887288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:58.887355 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:58.888051 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.888856 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.890069 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.890379 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.890459 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:58.896225 (+ 5766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:58.896227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:58.896227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:58.896237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:58.898741 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:58.899422 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:58.899423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:58.899424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:58.899488 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:58.899815 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:58.899817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:58.899817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:58.899970 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:58.900017 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:58.900499 (+ 482us) server_negotiation.cc:299] Negotiation successful
0925 22:02:58.900600 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:02:58.901178 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:58.886418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45271 (local address 127.9.112.66:60279)
0925 22:02:58.886605 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:58.886647 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:58.887156 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:58.887441 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:58.887446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:58.887658 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:58.887933 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:58.887939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.888964 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.888965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:58.889945 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:58.889948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:58.890548 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:58.890550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:58.890718 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:58.893334 (+ 2616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:58.893347 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:58.896060 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:58.898884 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:58.898897 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:58.898900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:58.899290 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:58.899561 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:58.899562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:58.899563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:58.899731 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:58.900066 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:58.900074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:58.900273 (+ 199us) client_negotiation.cc:769] Sending connection context
0925 22:02:58.900448 (+ 175us) client_negotiation.cc:240] Negotiation successful
0925 22:02:58.900633 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260925 22:02:58.901713 22740 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45271
I20260925 22:02:58.901844 22740 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.902072 22740 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
W20260925 22:02:58.902872 22740 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:45271 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 22:02:58.902952 22740 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.903038 22740 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:58.903508 22740 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:58.903586 22740 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
W20260925 22:02:58.903895 22740 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:02:59.009166 22743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:02:59.009492 22743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:02:59.009538 22743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:02:59.011590 22743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:02:59.011670 22743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:02:59.011720 22743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:02:59.014300 22743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.112.84:47149/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:42527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=41937
--tserver_master_addrs=127.9.112.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:02:59.014600 22743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:02:59.014901 22743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:02:59.022017 22743 server_base.cc:1034] running on GCE node
W20260925 22:02:59.021996 22748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:59.022368 22751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:02:59.022541 22749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:02:59.022794 22743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:02:59.023083 22743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:02:59.024253 22743 hybrid_clock.cc:648] HybridClock initialized: now 1790373779024204 us; error 64 us; skew 500 ppm
Sep 25 22:02:59 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373779, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:59.029152 22743 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:02:59.029790 22743 webserver.cc:460] Webserver started at http://127.9.112.67:41937/ using document root <none> and password file <none>
I20260925 22:02:59.030103 22743 fs_manager.cc:359] Metadata directory not provided
I20260925 22:02:59.030222 22743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 22:02:59 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373779, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:02:59.057933 22743 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:02:59.058959 22760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:59.059157 22743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:02:59.059257 22743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "5a0f400626004a9cb5ab483d6f11212e"
format_stamp: "Formatted at 2026-09-25 22:02:58 on dist-test-slave-794d"
server_key: "0a1d16b61fa777067bc1813d654aff06"
server_key_iv: "df90b21480444793f6e17928f86e8bfa"
server_key_version: "kuduclusterkey@0"
I20260925 22:02:59.059645 22743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:02:59.071671 22743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:02:59.073691 22743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:02:59.073843 22743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:02:59.074081 22743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:02:59.074419 22743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:02:59.074455 22743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:59.074486 22743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:02:59.074504 22743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:02:59.081261 22743 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:42527
I20260925 22:02:59.081651 22743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:02:59.082203 22873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:42527 every 8 connection(s)
I20260925 22:02:59.088683 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22743
Sep 25 22:02:59 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373779, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:02:59.096592 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:59.082393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:44875 (local address 127.9.112.126:45271)
0925 22:02:59.082446 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:59.082448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:59.082685 (+ 237us) server_negotiation.cc:407] Connection header received
0925 22:02:59.083207 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:59.083208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:59.083220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:59.083270 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:59.083965 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:59.084570 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:59.085652 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:59.085858 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:59.085915 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:59.093506 (+ 7591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:59.093507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:59.093508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:59.093514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:59.095104 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:59.095568 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:59.095569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:59.095569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:59.095615 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:59.095896 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:59.095897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:59.095897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:59.096011 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:59.096076 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:59.096406 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 22:02:59.096494 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:02:59.096813 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:59.082411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:45271 (local address 127.9.112.67:44875)
0925 22:02:59.082564 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 22:02:59.082605 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:02:59.083100 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:02:59.083343 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:02:59.083347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:02:59.083581 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:02:59.083860 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:59.083867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:59.084655 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:59.084656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:59.085563 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:02:59.085566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:02:59.085954 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:02:59.085955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:02:59.086085 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:59.091536 (+ 5451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:02:59.091550 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:02:59.093399 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:02:59.095180 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:59.095193 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:59.095195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:59.095498 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:59.095679 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:02:59.095679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:02:59.095680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:02:59.095832 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:02:59.096090 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:02:59.096099 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:02:59.096253 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 22:02:59.096381 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 22:02:59.096578 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260925 22:02:59.097334 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:02:59.089115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.9.112.126:45271)
0925 22:02:59.089264 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 22:02:59.089266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:02:59.089307 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:02:59.089361 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:02:59.089362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:02:59.089371 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:02:59.089435 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:02:59.090125 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:59.090677 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:59.091609 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:02:59.091845 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:02:59.091901 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:02:59.093285 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:02:59.093286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:02:59.093286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:02:59.093294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:02:59.094979 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:59.095398 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:59.095400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:59.095400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:59.095462 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:02:59.095630 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:02:59.095631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:02:59.095631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:02:59.095766 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:02:59.095943 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 22:02:59.097144 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0925 22:02:59.097239 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260925 22:02:59.097772 22874 heartbeater.cc:347] Connected to a master server at 127.9.112.126:45271
I20260925 22:02:59.097870 22874 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:59.098068 22874 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
W20260925 22:02:59.098691 22874 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:45271 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 22:02:59.098750 22874 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:59.098793 22874 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:59.099110 22874 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:59.099177 22874 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
W20260925 22:02:59.099474 22874 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:02:59.727666 22606 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:59.727828 22606 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:59.728304 22408 ts_manager.cc:194] Registered new tserver with Master: b60ee4544ea14742a4ee41967e4b5452 (127.9.112.65:39209)
I20260925 22:02:59.729380 22408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:60857
Sep 25 22:02:59 dist-test-slave-794d krb5kdc[21412](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790373779, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:02:59.904107 22740 heartbeater.cc:464] Registering TS with master...
I20260925 22:02:59.904273 22740 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:02:59.904707 22408 ts_manager.cc:194] Registered new tserver with Master: 6b104c5166f5483a8e750ba29519fec9 (127.9.112.66:37387)
I20260925 22:02:59.905231 22408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:60279
I20260925 22:03:00.099686 22874 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:00.099833 22874 heartbeater.cc:511] Master 127.9.112.126:45271 requested a full tablet report, sending...
I20260925 22:03:00.100251 22408 ts_manager.cc:194] Registered new tserver with Master: 5a0f400626004a9cb5ab483d6f11212e (127.9.112.67:42527)
I20260925 22:03:00.100827 22408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:44875
I20260925 22:03:00.108469 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:03:00.121256 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.112881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.9.112.126:45271)
0925 22:03:00.113245 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.113247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.113255 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:03:00.113282 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.113283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.113295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.114034 (+ 739us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:00.114076 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.114871 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.116047 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.116407 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.117855 (+ 1448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.117911 (+ 56us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:00.117912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:00.117912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:00.117917 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:00.119637 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:00.120031 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:00.120032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:00.120032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:00.120095 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:00.120306 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:00.120307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:00.120307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:00.120466 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:00.120990 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.121034 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.121119 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":80,"threads_started":1}
Sep 25 22:03:00 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373779, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:03:00.123813 22408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:03:00.126057 22408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:03:00.146449 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.129072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:39209 (local address 127.0.0.1:38008)
0925 22:03:00.133477 (+ 4405us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:00.133536 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:00.133611 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:00.134381 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:00.134384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:00.134386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:00.134468 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.134471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.136163 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.136165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.142749 (+ 6584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.142752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.145386 (+ 2634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.145387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.146166 (+ 779us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.146210 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 22:03:00.146288 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 22:03:00.146354 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4371,"thread_start_us":89,"threads_started":1}
I20260925 22:03:00.146929 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.129525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.9.112.65:39209)
0925 22:03:00.133320 (+ 3795us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.133327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.134157 (+ 830us) server_negotiation.cc:407] Connection header received
0925 22:03:00.134201 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.134204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.134222 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.134286 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:00.134546 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.136064 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.144303 (+ 8239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.145208 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.145260 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.145323 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.146789 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.146847 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3751,"thread_start_us":64,"threads_started":1}
I20260925 22:03:00.147996 22540 tablet_service.cc:1467] Processing CreateTablet for tablet ad6a0ba57df7414fa1c06dfce3e8461f (DEFAULT_TABLE table=test-table [id=96f51c6810204b6ba1c4cce01da637bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:00.148452 22540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6a0ba57df7414fa1c06dfce3e8461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:00.149717 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.129073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:42527 (local address 127.0.0.1:39926)
0925 22:03:00.137535 (+ 8462us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:00.137546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:00.138045 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:00.138423 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:00.138424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:00.138426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:00.138464 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.138465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.139774 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.139775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.145965 (+ 6190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.145968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.149448 (+ 3480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.149450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.149518 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.149523 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:03:00.149581 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 22:03:00.149627 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8440,"mutex_wait_us":5,"thread_start_us":91,"threads_started":1}
I20260925 22:03:00.150102 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.129344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.9.112.67:42527)
0925 22:03:00.133402 (+ 4058us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.133406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.137605 (+ 4199us) server_negotiation.cc:407] Connection header received
0925 22:03:00.138260 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.138263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.138279 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.138327 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:00.138528 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.139673 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.148355 (+ 8682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.149273 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.149331 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.149387 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.149977 (+ 590us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.150029 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4024,"thread_start_us":44,"threads_started":1}
I20260925 22:03:00.151381 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.129300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:37387 (local address 127.0.0.1:46986)
0925 22:03:00.137655 (+ 8355us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:00.137663 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:00.137700 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:00.138106 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:00.138107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:00.138108 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:00.138166 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.138168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.141522 (+ 3354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.141523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.144198 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.144201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.151117 (+ 6916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.151119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.151196 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.151201 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:03:00.151251 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 22:03:00.151296 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8345,"thread_start_us":61,"threads_started":1}
I20260925 22:03:00.152112 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.129398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.9.112.66:37387)
0925 22:03:00.137807 (+ 8409us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.137813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.137821 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:03:00.137856 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.137859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.137875 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.137943 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:00.140309 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.141426 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.147935 (+ 6509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.150938 (+ 3003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.151008 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.151068 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.151982 (+ 914us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.152036 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8369,"thread_start_us":47,"threads_started":1}
I20260925 22:03:00.152123 22808 tablet_service.cc:1467] Processing CreateTablet for tablet ad6a0ba57df7414fa1c06dfce3e8461f (DEFAULT_TABLE table=test-table [id=96f51c6810204b6ba1c4cce01da637bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:00.152514 22808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6a0ba57df7414fa1c06dfce3e8461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:00.156677 22674 tablet_service.cc:1467] Processing CreateTablet for tablet ad6a0ba57df7414fa1c06dfce3e8461f (DEFAULT_TABLE table=test-table [id=96f51c6810204b6ba1c4cce01da637bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:00.157068 22674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6a0ba57df7414fa1c06dfce3e8461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:00.161257 22910 tablet_bootstrap.cc:492] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: Bootstrap starting.
I20260925 22:03:00.161993 22910 tablet_bootstrap.cc:654] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:00.162393 22910 log.cc:824] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:00.162786 22909 tablet_bootstrap.cc:492] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Bootstrap starting.
I20260925 22:03:00.163671 22909 tablet_bootstrap.cc:654] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:00.163985 22909 log.cc:824] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:00.165072 22910 tablet_bootstrap.cc:492] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: No bootstrap required, opened a new log
I20260925 22:03:00.165163 22910 ts_tablet_manager.cc:1397] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 22:03:00.167444 22908 tablet_bootstrap.cc:492] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Bootstrap starting.
I20260925 22:03:00.168275 22908 tablet_bootstrap.cc:654] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:00.168164 22910 raft_consensus.cc:348] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.168381 22910 raft_consensus.cc:374] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:00.168427 22910 raft_consensus.cc:729] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60ee4544ea14742a4ee41967e4b5452, State: Initialized, Role: FOLLOWER
I20260925 22:03:00.168574 22910 consensus_queue.cc:260] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.168663 22908 log.cc:824] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:00.168881 22910 ts_tablet_manager.cc:1428] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 22:03:00.169014 22606 heartbeater.cc:503] Master 127.9.112.126:45271 was elected leader, sending a full tablet report...
I20260925 22:03:00.170998 22909 tablet_bootstrap.cc:492] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: No bootstrap required, opened a new log
I20260925 22:03:00.171097 22909 ts_tablet_manager.cc:1397] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260925 22:03:00.174172 22909 raft_consensus.cc:348] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.174378 22909 raft_consensus.cc:374] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:00.174424 22909 raft_consensus.cc:729] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b104c5166f5483a8e750ba29519fec9, State: Initialized, Role: FOLLOWER
I20260925 22:03:00.174556 22909 consensus_queue.cc:260] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.174851 22909 ts_tablet_manager.cc:1428] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260925 22:03:00.174976 22740 heartbeater.cc:503] Master 127.9.112.126:45271 was elected leader, sending a full tablet report...
I20260925 22:03:00.175716 22908 tablet_bootstrap.cc:492] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: No bootstrap required, opened a new log
I20260925 22:03:00.175804 22908 ts_tablet_manager.cc:1397] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260925 22:03:00.179081 22908 raft_consensus.cc:348] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.179311 22908 raft_consensus.cc:374] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:00.179356 22908 raft_consensus.cc:729] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0f400626004a9cb5ab483d6f11212e, State: Initialized, Role: FOLLOWER
I20260925 22:03:00.179545 22908 consensus_queue.cc:260] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.179888 22908 ts_tablet_manager.cc:1428] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 22:03:00.180019 22874 heartbeater.cc:503] Master 127.9.112.126:45271 was elected leader, sending a full tablet report...
W20260925 22:03:00.210069 22607 tablet.cc:2369] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:03:00.333140 22875 tablet.cc:2369] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:03:00.387077 22741 tablet.cc:2369] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 22:03:00 dist-test-slave-794d krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373779, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for HTTP/127.9.112.84@KRBTEST.COM
I20260925 22:03:00.499639 22915 raft_consensus.cc:482] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:00.499853 22915 raft_consensus.cc:504] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.500448 22915 leader_election.cc:288] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60ee4544ea14742a4ee41967e4b5452 (127.9.112.65:39209), 5a0f400626004a9cb5ab483d6f11212e (127.9.112.67:42527)
I20260925 22:03:00.506546 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.501061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:48725 (local address 127.9.112.67:42527)
0925 22:03:00.501123 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.501126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.501494 (+ 368us) server_negotiation.cc:407] Connection header received
0925 22:03:00.501607 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.501608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.501620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.501649 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:00.501912 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.502737 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.505439 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.506046 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.506108 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.506121 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.506393 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.506449 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:03:00.506889 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.500738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:42527 (local address 127.9.112.66:48725)
0925 22:03:00.501423 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:00.501435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:00.501526 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:00.501726 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:00.501726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:00.501728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:00.501824 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.501827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.502844 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.502845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.505285 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.505292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.506228 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.506229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.506297 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.506301 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:00.506710 (+ 409us) client_negotiation.cc:240] Negotiation successful
0925 22:03:00.506769 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":156,"threads_started":1}
I20260925 22:03:00.507302 22828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a0ba57df7414fa1c06dfce3e8461f" candidate_uuid: "6b104c5166f5483a8e750ba29519fec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0f400626004a9cb5ab483d6f11212e" is_pre_election: true
I20260925 22:03:00.507593 22828 raft_consensus.cc:2454] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b104c5166f5483a8e750ba29519fec9 in term 0.
I20260925 22:03:00.507900 22628 leader_election.cc:302] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a0f400626004a9cb5ab483d6f11212e, 6b104c5166f5483a8e750ba29519fec9; no voters:
I20260925 22:03:00.508062 22915 raft_consensus.cc:2781] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:03:00.508108 22915 raft_consensus.cc:482] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:03:00.508129 22915 raft_consensus.cc:3037] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:00.509356 22915 raft_consensus.cc:504] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.509526 22915 leader_election.cc:288] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [CANDIDATE]: Term 1 election: Requested vote from peers b60ee4544ea14742a4ee41967e4b5452 (127.9.112.65:39209), 5a0f400626004a9cb5ab483d6f11212e (127.9.112.67:42527)
I20260925 22:03:00.509770 22828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a0ba57df7414fa1c06dfce3e8461f" candidate_uuid: "6b104c5166f5483a8e750ba29519fec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a0f400626004a9cb5ab483d6f11212e"
I20260925 22:03:00.509867 22828 raft_consensus.cc:3037] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:00.510914 22828 raft_consensus.cc:2454] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b104c5166f5483a8e750ba29519fec9 in term 1.
I20260925 22:03:00.511396 22628 leader_election.cc:302] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a0f400626004a9cb5ab483d6f11212e, 6b104c5166f5483a8e750ba29519fec9; no voters:
I20260925 22:03:00.511485 22915 raft_consensus.cc:2781] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:00.511677 22915 raft_consensus.cc:686] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 1 LEADER]: Becoming Leader. State: Replica: 6b104c5166f5483a8e750ba29519fec9, State: Running, Role: LEADER
I20260925 22:03:00.511809 22915 consensus_queue.cc:237] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } }
I20260925 22:03:00.512988 22408 catalog_manager.cc:5495] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b104c5166f5483a8e750ba29519fec9 (127.9.112.66). New cstate: current_term: 1 leader_uuid: "6b104c5166f5483a8e750ba29519fec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b104c5166f5483a8e750ba29519fec9" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 37387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 } health_report { overall_health: UNKNOWN } } }
I20260925 22:03:00.514927 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.501104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:39209 (local address 127.9.112.66:48695)
0925 22:03:00.511350 (+ 10246us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:00.511364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:00.511457 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:00.511687 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:00.511689 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:00.511691 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:00.511733 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.511735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.512691 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.512692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.513802 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:00.513806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.514691 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.514692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:00.514750 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.514754 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:00.514805 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 22:03:00.514854 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10227,"thread_start_us":87,"threads_started":1}
I20260925 22:03:00.515055 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.501337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:48695 (local address 127.9.112.65:39209)
0925 22:03:00.501401 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.501404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.511421 (+ 10017us) server_negotiation.cc:407] Connection header received
0925 22:03:00.511561 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.511563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.511577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.511611 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:00.511807 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.512533 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.514014 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.514572 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.514646 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.514659 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.514916 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.514979 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:03:00.515645 22544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a0ba57df7414fa1c06dfce3e8461f" candidate_uuid: "6b104c5166f5483a8e750ba29519fec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60ee4544ea14742a4ee41967e4b5452" is_pre_election: true
I20260925 22:03:00.515650 22560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a0ba57df7414fa1c06dfce3e8461f" candidate_uuid: "6b104c5166f5483a8e750ba29519fec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60ee4544ea14742a4ee41967e4b5452"
I20260925 22:03:00.515842 22544 raft_consensus.cc:2454] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b104c5166f5483a8e750ba29519fec9 in term 0.
I20260925 22:03:00.580736 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.576896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.9.112.66:37387)
0925 22:03:00.576962 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:00.576964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:00.576974 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:00.577054 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:00.577056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:00.577084 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:00.577163 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:00.577364 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.578141 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.579410 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:00.579740 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:00.579809 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:00.580044 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:00.580218 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:03:00.580306 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:00.580599 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 22:03:00.580632 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:03:00.583232 22560 raft_consensus.cc:3037] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:00.583324 22828 raft_consensus.cc:1261] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 1 FOLLOWER]: Refusing update from remote peer 6b104c5166f5483a8e750ba29519fec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:00.583719 22921 consensus_queue.cc:1035] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a0f400626004a9cb5ab483d6f11212e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:00.584529 22560 raft_consensus.cc:1261] T ad6a0ba57df7414fa1c06dfce3e8461f P b60ee4544ea14742a4ee41967e4b5452 [term 1 FOLLOWER]: Refusing update from remote peer 6b104c5166f5483a8e750ba29519fec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:00.585047 22921 consensus_queue.cc:1035] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60ee4544ea14742a4ee41967e4b5452" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:00.589775 22924 mvcc.cc:205] Tried to move back new op lower bound from 7333371005265813504 to 7333371004977123328. Current Snapshot: MvccSnapshot[applied={T|T < 7333371005265813504}]
I20260925 22:03:00.594818 22927 mvcc.cc:205] Tried to move back new op lower bound from 7333371005265813504 to 7333371004977123328. Current Snapshot: MvccSnapshot[applied={T|T < 7333371005265813504}]
I20260925 22:03:00.596001 22925 mvcc.cc:205] Tried to move back new op lower bound from 7333371005265813504 to 7333371004977123328. Current Snapshot: MvccSnapshot[applied={T|T < 7333371005265813504}]
I20260925 22:03:00.597265 22408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:00.597400 22408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:00.598242 22408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9c68c780a424db99c09c9b6ce81b318: Sending DeleteTablet for 3 replicas of tablet ad6a0ba57df7414fa1c06dfce3e8461f
I20260925 22:03:00.598646 22808 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6a0ba57df7414fa1c06dfce3e8461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:00 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:39926
I20260925 22:03:00.599071 22540 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6a0ba57df7414fa1c06dfce3e8461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:00 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:38008
I20260925 22:03:00.599426 22674 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6a0ba57df7414fa1c06dfce3e8461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:00 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:46986
I20260925 22:03:00.599977 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22446
I20260925 22:03:00.603421 22935 tablet_replica.cc:330] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: stopping tablet replica
I20260925 22:03:00.603566 22935 raft_consensus.cc:2229] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:03:00.603688 22935 raft_consensus.cc:2258] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:00.608775 22935 ts_tablet_manager.cc:1905] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:00.611195 22935 ts_tablet_manager.cc:1918] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:00.611292 22935 log.cc:1196] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/ad6a0ba57df7414fa1c06dfce3e8461f
I20260925 22:03:00.611593 22935 ts_tablet_manager.cc:1939] T ad6a0ba57df7414fa1c06dfce3e8461f P 6b104c5166f5483a8e750ba29519fec9: Deleting consensus metadata
I20260925 22:03:00.610509 22933 tablet_replica.cc:330] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: stopping tablet replica
I20260925 22:03:00.611868 22933 raft_consensus.cc:2229] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:00.612066 22933 raft_consensus.cc:2258] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:00.612715 22933 ts_tablet_manager.cc:1905] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:00.615213 22395 catalog_manager.cc:4841] TS 6b104c5166f5483a8e750ba29519fec9 (127.9.112.66:37387): tablet ad6a0ba57df7414fa1c06dfce3e8461f (table test-table [id=96f51c6810204b6ba1c4cce01da637bc]) successfully deleted
W20260925 22:03:00.615629 22395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:39209 (error 108)
I20260925 22:03:00.619493 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22610
I20260925 22:03:00.628227 22933 ts_tablet_manager.cc:1918] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:00.628399 22933 log.cc:1196] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/ad6a0ba57df7414fa1c06dfce3e8461f
I20260925 22:03:00.628756 22933 ts_tablet_manager.cc:1939] T ad6a0ba57df7414fa1c06dfce3e8461f P 5a0f400626004a9cb5ab483d6f11212e: Deleting consensus metadata
I20260925 22:03:00.629417 22393 catalog_manager.cc:4841] TS 5a0f400626004a9cb5ab483d6f11212e (127.9.112.67:42527): tablet ad6a0ba57df7414fa1c06dfce3e8461f (table test-table [id=96f51c6810204b6ba1c4cce01da637bc]) successfully deleted
I20260925 22:03:00.630450 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22743
I20260925 22:03:00.630491 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:00.630301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:39209 (local address 127.0.0.1:38020)
0925 22:03:00.630381 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:00.630444 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:39209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 22:03:00.631404 22395 catalog_manager.cc:4584] TS b60ee4544ea14742a4ee41967e4b5452 (127.9.112.65:39209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad6a0ba57df7414fa1c06dfce3e8461f: Network error: Client connection negotiation failed: client connection to 127.9.112.65:39209: connect: Connection refused (error 111)
I20260925 22:03:00.640533 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22375
I20260925 22:03:00.648811 9665 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 22:03:00.983906 9665 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 22:03:00.984028 9665 mini_ranger.cc:66] Stopping Ranger...
I20260925 22:03:01.321074 9665 mini_ranger.cc:68] Stopped Ranger
2026-09-25 22:03:01.321 UTC [21442] LOG: received smart shutdown request
2026-09-25 22:03:01.325 UTC [21442] LOG: background worker "logical replication launcher" (PID 21451) exited with exit code 1
2026-09-25 22:03:01.326 UTC [21446] LOG: shutting down
2026-09-25 22:03:01.584 UTC [21442] LOG: database system is shut down
2026-09-25T22:03:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51586 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:03:01 dist-test-slave-794d krb5kdc[22945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:03:01 dist-test-slave-794d krb5kdc[22945](info): set up 2 sockets
Sep 25 22:03:01 dist-test-slave-794d krb5kdc[22945](info): commencing operation
krb5kdc: starting...
W20260925 22:03:02.660729 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:03:02 dist-test-slave-794d krb5kdc[22945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:03:02.792419 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:42849
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:42849,127.9.112.125:43043,127.9.112.124:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:02.882130 22958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:02.882396 22958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:02.882431 22958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:02.883682 22958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:02.883734 22958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:02.883764 22958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:02.883780 22958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:02.883796 22958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:02.885510 22958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:42849,127.9.112.125:43043,127.9.112.124:38153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:42849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:02.885838 22958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:02.886087 22958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:02.891417 22963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:02.891474 22958 server_base.cc:1034] running on GCE node
W20260925 22:03:02.891439 22964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:02.891434 22966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:02.891939 22958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:02.892268 22958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:02.893468 22958 hybrid_clock.cc:648] HybridClock initialized: now 1790373782893441 us; error 47 us; skew 500 ppm
Sep 25 22:03:02 dist-test-slave-794d krb5kdc[22945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:02.896308 22958 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:02.896798 22958 webserver.cc:460] Webserver started at http://127.9.112.126:39045/ using document root <none> and password file <none>
I20260925 22:03:02.897027 22958 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:02.897070 22958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:02.897178 22958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:02.898402 22958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "a38652de7c9945838635f0d374c7e2d3"
format_stamp: "Formatted at 2026-09-25 22:03:02 on dist-test-slave-794d"
server_key: "adb9de53f260b22039899d58466bee3e"
server_key_iv: "1c03702418d1feeae15d9ed0a7bb6d4d"
server_key_version: "encryptionkey@0"
I20260925 22:03:02.898736 22958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "a38652de7c9945838635f0d374c7e2d3"
format_stamp: "Formatted at 2026-09-25 22:03:02 on dist-test-slave-794d"
server_key: "adb9de53f260b22039899d58466bee3e"
server_key_iv: "1c03702418d1feeae15d9ed0a7bb6d4d"
server_key_version: "encryptionkey@0"
I20260925 22:03:02.900113 22958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:03:02.900923 22973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:02.901085 22958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:02.901166 22958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "a38652de7c9945838635f0d374c7e2d3"
format_stamp: "Formatted at 2026-09-25 22:03:02 on dist-test-slave-794d"
server_key: "adb9de53f260b22039899d58466bee3e"
server_key_iv: "1c03702418d1feeae15d9ed0a7bb6d4d"
server_key_version: "encryptionkey@0"
I20260925 22:03:02.901243 22958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:02.922752 22958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:02.925853 22958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:02.926050 22958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:02.930780 22958 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:42849
I20260925 22:03:02.930823 23025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:42849 every 8 connection(s)
I20260925 22:03:02.931213 22958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:02.932255 23026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:02.934376 23026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:03:02.938750 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 22958
I20260925 22:03:02.938879 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:03:02.939097 9665 external_mini_cluster.cc:1428] Setting key 8793f479d84a980a13a3b7726c41c414
Sep 25 22:03:02 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:02.945444 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:02.935298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42849 (local address 127.0.0.1:34992)
0925 22:03:02.935511 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:02.935553 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:02.936079 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:02.936427 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:02.936440 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:02.936737 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:02.936947 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:02.936952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:02.938075 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:02.938076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:02.939036 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:02.939039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:02.939621 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:02.939622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:02.939706 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:02.940328 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:02.940367 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:02.942087 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:02.943929 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:02.943935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:02.943938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:02.944136 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:02.944367 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:02.944368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:02.944368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:02.944453 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:02.944807 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:02.944812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:02.944910 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 22:03:02.944995 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 22:03:02.945222 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":70,"thread_start_us":99,"threads_started":1}
I20260925 22:03:02.945442 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:02.935326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34992 (local address 127.9.112.126:42849)
0925 22:03:02.935516 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:02.935525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:02.935662 (+ 137us) server_negotiation.cc:407] Connection header received
0925 22:03:02.936249 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:02.936251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:02.936271 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:02.936409 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:02.937049 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:02.937974 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:02.939130 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:02.939530 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:02.939623 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:02.942192 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:02.942199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:02.942199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:02.942206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:02.943838 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:02.944212 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:02.944216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:02.944217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:02.944297 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:02.944506 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:02.944506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:02.944507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:02.944749 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:02.944822 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:02.945027 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 22:03:02.945149 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260925 22:03:02.946254 23026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:03:02.946709 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:02.946552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43738)
0925 22:03:02.946629 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:02.946682 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 22:03:02.946866 22974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.112.125@KRBTEST.COM; defaulting to no policy
I20260925 22:03:02.947309 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:02.947224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43742)
0925 22:03:02.947260 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:02.947285 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 22:03:02.947422 23026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:43043: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab.
Entry for principal kudu/127.9.112.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab.
I20260925 22:03:02.968037 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:43043
--webserver_interface=127.9.112.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab
--principal=kudu/127.9.112.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:42849,127.9.112.125:43043,127.9.112.124:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:02.996693 23026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } attempt: 1
I20260925 22:03:02.997418 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:02.997228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43748)
0925 22:03:02.997297 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:02.997348 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 22:03:02.997880 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:02.997683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43760)
0925 22:03:02.997771 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:02.997844 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260925 22:03:02.997994 23026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:43043: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
W20260925 22:03:03.061595 23033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:03.061864 23033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:03.061887 23033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:03.063156 23033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:03.063206 23033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:03.063220 23033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:03.063238 23033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:03.063251 23033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:03.064989 23033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:42849,127.9.112.125:43043,127.9.112.124:38153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab
--principal=kudu/127.9.112.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:43043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.112.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:03.065277 23033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:03.065516 23033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:03:03.066165 23026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } attempt: 2
I20260925 22:03:03.066805 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.066580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43764)
0925 22:03:03.066674 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.066756 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 22:03:03.067240 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.067103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43780)
0925 22:03:03.067163 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.067200 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 22:03:03.067353 23026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:43043: Network error: Client connection negotiation failed: client connection to 127.9.112.125:43043: connect: Connection refused (error 111)
W20260925 22:03:03.070817 23039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:03.070768 23038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:03.070775 23041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:03.071094 23033 server_base.cc:1034] running on GCE node
I20260925 22:03:03.071448 23033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:03.071790 23033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:03.072932 23033 hybrid_clock.cc:648] HybridClock initialized: now 1790373783072916 us; error 29 us; skew 500 ppm
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:03.075695 23033 init.cc:377] Logged in from keytab as kudu/127.9.112.125@KRBTEST.COM (short username kudu)
I20260925 22:03:03.076195 23033 webserver.cc:460] Webserver started at http://127.9.112.125:42341/ using document root <none> and password file <none>
I20260925 22:03:03.076479 23033 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:03.076534 23033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:03.076676 23033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:03.077854 23033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data/instance:
uuid: "81c153b5e61d4f358438810010e682d6"
format_stamp: "Formatted at 2026-09-25 22:03:03 on dist-test-slave-794d"
server_key: "80dd6791b45a3383a26d26a7996586a7"
server_key_iv: "b7a4288b67cb4018f4bb9db94169126c"
server_key_version: "encryptionkey@0"
I20260925 22:03:03.078354 23033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal/instance:
uuid: "81c153b5e61d4f358438810010e682d6"
format_stamp: "Formatted at 2026-09-25 22:03:03 on dist-test-slave-794d"
server_key: "80dd6791b45a3383a26d26a7996586a7"
server_key_iv: "b7a4288b67cb4018f4bb9db94169126c"
server_key_version: "encryptionkey@0"
I20260925 22:03:03.079838 23033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:03:03.080757 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:03.080961 23033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:03.081060 23033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal
uuid: "81c153b5e61d4f358438810010e682d6"
format_stamp: "Formatted at 2026-09-25 22:03:03 on dist-test-slave-794d"
server_key: "80dd6791b45a3383a26d26a7996586a7"
server_key_iv: "b7a4288b67cb4018f4bb9db94169126c"
server_key_version: "encryptionkey@0"
I20260925 22:03:03.081131 23033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:03.106801 23033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:03.109179 23033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:03.109347 23033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:03.113930 23033 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.125:43043
I20260925 22:03:03.113991 23100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.125:43043 every 8 connection(s)
I20260925 22:03:03.114413 23033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
I20260925 22:03:03.115480 23101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:03.117424 23101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:03.124651 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23033
I20260925 22:03:03.124782 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-1/wal/instance
I20260925 22:03:03.124987 9665 external_mini_cluster.cc:1428] Setting key aaf74dbb9e7019a988470c8db34fac8d
I20260925 22:03:03.127210 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.118250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.9.112.126:42849)
0925 22:03:03.118348 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.118350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.118605 (+ 255us) server_negotiation.cc:407] Connection header received
0925 22:03:03.119165 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.119166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.119185 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.119241 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.119976 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.120596 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.121642 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.121870 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.121931 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.124190 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.124191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.124192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.124200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.125808 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.126250 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.126251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.126252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.126311 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.126542 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.126543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.126544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.126668 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.126725 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.127010 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.127108 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 22:03:03.127415 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.118207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42849 (local address 127.0.0.1:35004)
0925 22:03:03.118435 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.118482 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.119048 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.119329 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.119332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.119617 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.119867 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.119872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.120687 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.120688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.121559 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.121562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.121944 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.121945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.122074 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.122616 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.122626 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.124096 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.125914 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.125923 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.125925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.126180 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.126371 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.126372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.126372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.126486 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.126724 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.126732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.126889 (+ 157us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.126983 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.127155 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260925 22:03:03.128005 23101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.124@KRBTEST.COM; defaulting to no policy
I20260925 22:03:03.136047 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.128297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43792)
0925 22:03:03.128379 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.128389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.128424 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.128680 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.128680 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.128824 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.128890 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.128892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.129802 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.129803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.130592 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.130595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.131194 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.131195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.131282 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.131736 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.131741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.133208 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.135021 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.135022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.135022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.135177 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.135353 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.135353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.135354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.135408 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.135726 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.135727 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.135781 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.135839 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.135934 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 22:03:03.136106 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.128377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.9.112.125:43043)
0925 22:03:03.128542 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.128547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.128567 (+ 20us) server_negotiation.cc:407] Connection header received
0925 22:03:03.128586 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.128587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.128598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.128665 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.128952 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.129715 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.130671 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.131095 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.131167 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.133288 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.133298 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.133299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.133306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.134948 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.135244 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.135248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.135248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.135301 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.135455 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.135456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.135456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.135618 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.135698 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.135884 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.135985 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.124@KRBTEST.COM" created.
I20260925 22:03:03.136767 23101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:03:03.137109 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.137006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48614)
0925 22:03:03.137045 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.137091 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 22:03:03.137234 23051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
I20260925 22:03:03.137660 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.137575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48626)
0925 22:03:03.137615 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.137640 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 22:03:03.137754 23101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38153: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab.
Entry for principal kudu/127.9.112.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab.
I20260925 22:03:03.154035 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:38153
--webserver_interface=127.9.112.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab
--principal=kudu/127.9.112.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:42849,127.9.112.125:43043,127.9.112.124:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:03.158516 23026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } attempt: 3
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:03:03.167121 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.158988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43808)
0925 22:03:03.159052 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.159066 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.159114 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.159283 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.159284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.159529 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.159637 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.159642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.160716 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.160717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.161289 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.161292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.161804 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.161804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.161870 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.162416 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.162421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.163968 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.166092 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.166094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.166094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.166293 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.166499 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.166500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.166500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.166565 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.166822 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.166823 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.166875 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.166931 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.167032 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:03:03.167153 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.159090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.9.112.125:43043)
0925 22:03:03.159164 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.159167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.159177 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:03.159206 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.159212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.159225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.159270 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.159760 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.160618 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.161381 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.161728 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.161810 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.164065 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.164068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.164068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.164078 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.165993 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.166361 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.166362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.166363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.166431 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.166619 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.166619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.166620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.166761 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.166822 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.166965 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.167049 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:03:03.167685 23026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:03:03.168116 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.167976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48636)
0925 22:03:03.168031 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.168076 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 22:03:03.168471 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.168399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48646)
0925 22:03:03.168427 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.168452 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260925 22:03:03.168598 23026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38153: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
I20260925 22:03:03.186998 23101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } attempt: 1
I20260925 22:03:03.187620 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.187474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48650)
0925 22:03:03.187538 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.187578 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:03:03.187978 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.187889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48664)
0925 22:03:03.187928 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.187954 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260925 22:03:03.188066 23101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38153: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
I20260925 22:03:03.199765 23026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } attempt: 1
I20260925 22:03:03.200484 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.200301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48676)
0925 22:03:03.200385 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.200440 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 22:03:03.200906 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.200806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48684)
0925 22:03:03.200853 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.200881 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 22:03:03.201018 23026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38153: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
W20260925 22:03:03.249195 23108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:03.249465 23108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:03.249488 23108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:03.250758 23108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:03.250810 23108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:03.250823 23108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:03.250835 23108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:03.250845 23108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:03.252525 23108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:42849,127.9.112.125:43043,127.9.112.124:38153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab
--principal=kudu/127.9.112.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:38153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.112.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:03.252807 23108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:03.253052 23108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:03:03.256232 23101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } attempt: 2
I20260925 22:03:03.256814 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.256654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48694)
0925 22:03:03.256720 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.256763 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 22:03:03.257521 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.257340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48704)
0925 22:03:03.257442 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.257482 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260925 22:03:03.257675 23101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38153: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
W20260925 22:03:03.258970 23114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:03.258993 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:03.258997 23116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:03.259130 23108 server_base.cc:1034] running on GCE node
I20260925 22:03:03.259562 23108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:03.259902 23108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:03.261081 23108 hybrid_clock.cc:648] HybridClock initialized: now 1790373783261057 us; error 35 us; skew 500 ppm
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:03.263973 23108 init.cc:377] Logged in from keytab as kudu/127.9.112.124@KRBTEST.COM (short username kudu)
I20260925 22:03:03.264519 23108 webserver.cc:460] Webserver started at http://127.9.112.124:43071/ using document root <none> and password file <none>
I20260925 22:03:03.264731 23108 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:03.264772 23108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:03.264889 23108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:03.266064 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data/instance:
uuid: "2de4c74811934ef9ad459c662c4f20de"
format_stamp: "Formatted at 2026-09-25 22:03:03 on dist-test-slave-794d"
server_key: "2d8e206e3ceeb56b0d766005d6f6b504"
server_key_iv: "b28bf997a6cdaae73e77d019a4a54951"
server_key_version: "encryptionkey@0"
I20260925 22:03:03.266393 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal/instance:
uuid: "2de4c74811934ef9ad459c662c4f20de"
format_stamp: "Formatted at 2026-09-25 22:03:03 on dist-test-slave-794d"
server_key: "2d8e206e3ceeb56b0d766005d6f6b504"
server_key_iv: "b28bf997a6cdaae73e77d019a4a54951"
server_key_version: "encryptionkey@0"
I20260925 22:03:03.267817 23108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:03:03.268574 23123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:03.268754 23108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:03.268846 23108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal
uuid: "2de4c74811934ef9ad459c662c4f20de"
format_stamp: "Formatted at 2026-09-25 22:03:03 on dist-test-slave-794d"
server_key: "2d8e206e3ceeb56b0d766005d6f6b504"
server_key_iv: "b28bf997a6cdaae73e77d019a4a54951"
server_key_version: "encryptionkey@0"
I20260925 22:03:03.268909 23108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:03.276211 23026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } attempt: 2
I20260925 22:03:03.276921 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.276744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48720)
0925 22:03:03.276826 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.276877 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 22:03:03.277318 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.277217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48724)
0925 22:03:03.277264 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.277291 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 22:03:03.277415 23026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38153: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38153: connect: Connection refused (error 111)
I20260925 22:03:03.280913 23108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:03.283207 23108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:03.283408 23108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:03.288054 23108 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.124:38153
I20260925 22:03:03.288079 23175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.124:38153 every 8 connection(s)
I20260925 22:03:03.288533 23108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
I20260925 22:03:03.289608 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:03.290153 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23108
I20260925 22:03:03.290246 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790373628768857-9665-0/minicluster-data/master-2/wal/instance
I20260925 22:03:03.290444 9665 external_mini_cluster.cc:1428] Setting key 07a40a4416c49f41275c4a2ffcdc9f2e
I20260925 22:03:03.291704 23176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:03.302047 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.291041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.9.112.124:38153)
0925 22:03:03.291207 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.291216 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.291244 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:03:03.291710 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.291718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.291770 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.291897 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.292695 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.293648 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.294431 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.295142 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.295318 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.298166 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.298176 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.298176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.298184 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.300114 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.300612 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.300616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.300617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.300733 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.300938 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.300939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.300939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.301317 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.301434 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.301657 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.301809 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260925 22:03:03.302093 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.292517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.9.112.126:42849)
0925 22:03:03.292577 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.292579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.293746 (+ 1167us) server_negotiation.cc:407] Connection header received
0925 22:03:03.293780 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.293781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.293795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.293850 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.294397 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.294864 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.295773 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.295980 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.296037 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.298624 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.298626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.298626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.298636 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.300752 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.301159 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.301160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.301161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.301232 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.301487 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.301488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.301488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.301627 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.301695 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.301900 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.301993 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:03:03.302137 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.292413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42849 (local address 127.0.0.1:35010)
0925 22:03:03.292546 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.292587 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.292652 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.293900 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.293909 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.294180 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.294310 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.294312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.294958 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.294959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.295703 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.295705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.296078 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.296079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.296154 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.296803 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.296812 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.298509 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.300865 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.300872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.300875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.301087 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.301323 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.301327 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.301327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.301407 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.301703 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.301711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.301766 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.301850 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.301995 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260925 22:03:03.302846 23176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:03.311221 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.303115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:43043 (local address 127.0.0.1:43812)
0925 22:03:03.303156 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.303164 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.303192 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.303359 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.303360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.303501 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.303557 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.303559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.304616 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.304617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.305451 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.305454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.305985 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.305986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.306056 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.306651 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.306657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.308112 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.310122 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.310122 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.310123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.310305 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.310526 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.310527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.310528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.310596 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.310888 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.310890 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.310946 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.311034 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.311135 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 22:03:03.311244 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.303191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.9.112.125:43043)
0925 22:03:03.303242 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.303244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.303251 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:03:03.303286 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.303287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.303296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.303339 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.303625 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.304519 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.305553 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.305883 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.305988 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.308210 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.308212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.308213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.308221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.310031 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.310374 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.310375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.310376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.310445 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.310666 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.310666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.310666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.310805 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.310868 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.311062 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.311145 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:03:03.311751 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.302986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.9.112.126:42849)
0925 22:03:03.303028 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.303030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.303078 (+ 48us) server_negotiation.cc:407] Connection header received
0925 22:03:03.303154 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.303154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.303165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.303218 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.303778 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.304537 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.305597 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.305934 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.305993 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.308819 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.308821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.308821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.308829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.310370 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.310746 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.310747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.310747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.310799 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.311035 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.311035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.311036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.311172 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.311520 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.311567 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.311651 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:03:03.312112 23176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:03:03.319110 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.312446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48736)
0925 22:03:03.312480 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.312492 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.312529 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.312653 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.312654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.312800 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.312853 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.312855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.313640 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.313641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.314274 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.314276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.314714 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.314715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.314765 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.315210 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.315214 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.316656 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.318113 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.318114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.318115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.318332 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.318564 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.318565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.318565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.318629 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.318827 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.318828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.318872 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.318939 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.319036 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 22:03:03.319199 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.312526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.9.112.124:38153)
0925 22:03:03.312560 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.312562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.312568 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:03:03.312585 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.312585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.312593 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.312641 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.312922 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.313567 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.314356 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.314657 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.314707 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.316737 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.316739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.316740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.316748 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.318030 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.318456 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.318457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.318457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.318508 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.318682 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.318683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.318683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.318780 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.318832 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.318969 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.319094 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":18}
I20260925 22:03:03.320315 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.312486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.9.112.125:43043)
0925 22:03:03.312575 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.312576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.312628 (+ 52us) server_negotiation.cc:407] Connection header received
0925 22:03:03.312700 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.312701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.312711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.312793 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.313303 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.313836 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.314568 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.314797 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.314872 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.317299 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.317300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.317301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.317309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.318830 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.319556 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.319557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.319557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.319613 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.319890 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.319890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.319891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.319999 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.320165 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.320191 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.320248 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 22:03:03.321287 23176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: Bootstrap starting.
I20260925 22:03:03.322446 23176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:03.322630 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:03:03.322799 23176 log.cc:824] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:03.323724 23176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: No bootstrap required, opened a new log
I20260925 22:03:03.326104 23176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.326468 23176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:03.326550 23176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de4c74811934ef9ad459c662c4f20de, State: Initialized, Role: FOLLOWER
I20260925 22:03:03.326726 23176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.327045 23189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.327193 23189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:03:03.327113 23176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: configured and running, proceeding with master startup.
W20260925 22:03:03.329902 23200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:03:03.330019 23200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:03:03.332285 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.324462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.9.112.125:43043)
0925 22:03:03.324499 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.324501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.324602 (+ 101us) server_negotiation.cc:407] Connection header received
0925 22:03:03.324687 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.324688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.324698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.324726 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.325516 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.326285 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.327325 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.327621 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.327682 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.329480 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.329482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.329482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.329489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.330973 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.331340 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.331342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.331342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.331397 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.331667 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.331668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.331668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.331807 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.331898 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.332105 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.332187 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:03:03.332762 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.324216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.9.112.124:38153)
0925 22:03:03.324255 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.324257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.324592 (+ 335us) server_negotiation.cc:407] Connection header received
0925 22:03:03.324687 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.324688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.324698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.324726 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.325515 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.326302 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.328051 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.328274 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.328333 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.330050 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.330051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.330052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.330059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.331577 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.331896 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.331897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.331897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.331967 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.332166 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.332167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.332167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.332322 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.332415 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.332609 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.332682 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:03:03.329656 23108 master_runner.cc:186] Error getting master registration for 127.9.112.126:42849: OK, instance_id { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" instance_seqno: 1790373782929420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:03:03.333642 23108 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:03:03.337112 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.324242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.9.112.126:42849)
0925 22:03:03.324286 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.324288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.324577 (+ 289us) server_negotiation.cc:407] Connection header received
0925 22:03:03.324683 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.324685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.324698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.324729 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.325560 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.326270 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.327368 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.327643 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.327686 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.334417 (+ 6731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.334419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.334419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.334428 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.335772 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.336177 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.336178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.336178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.336239 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.336590 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.336591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.336592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.336731 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.336825 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.336966 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.337039 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:03:03.348850 23101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } attempt: 3
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373783, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
I20260925 22:03:03.356815 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.349309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.9.112.124:38153)
0925 22:03:03.349364 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.349366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.349381 (+ 15us) server_negotiation.cc:407] Connection header received
0925 22:03:03.349462 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.349463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.349474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.349521 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.349858 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.350366 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.351035 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.351264 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.351323 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.353842 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.353844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.353845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.353889 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.355483 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.355862 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.355864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.355864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.355923 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.356093 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.356094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.356094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.356227 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.356448 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.356614 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.356695 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:03:03.356822 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.349233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48748)
0925 22:03:03.349299 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.349312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.349400 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.349524 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.349525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.349727 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.349785 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.349787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.350435 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.350436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.350961 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.350963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.351327 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.351328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.351399 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.351949 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.351955 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.353747 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.355581 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.355582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.355583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.355770 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.355979 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.355980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.355981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.356041 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.356282 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.356283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.356488 (+ 205us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.356563 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.356692 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 22:03:03.358618 23101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: Bootstrap starting.
I20260925 22:03:03.359670 23101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:03.359993 23101 log.cc:824] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:03.362186 23101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: No bootstrap required, opened a new log
I20260925 22:03:03.364463 23101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.364642 23101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:03.364679 23101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c153b5e61d4f358438810010e682d6, State: Initialized, Role: FOLLOWER
I20260925 22:03:03.364818 23101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.365141 23203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.365206 23101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:03.365232 23203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:03:03.367172 23214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:03:03.367316 23214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:03:03.367957 23033 master_runner.cc:186] Error getting master registration for 127.9.112.126:42849: OK, instance_id { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" instance_seqno: 1790373782929420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:03:03.370177 23033 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:03:03.376598 23026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } attempt: 3
Sep 25 22:03:03 dist-test-slave-794d krb5kdc[22945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
I20260925 22:03:03.383770 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.377032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38153 (local address 127.0.0.1:48760)
0925 22:03:03.377102 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:03.377115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:03.377166 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:03.377309 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:03.377310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:03.377515 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:03.377574 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.377577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.378230 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.378232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.378760 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:03.378763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.379112 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.379114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:03.379171 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.379659 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:03.379665 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:03.381049 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:03.382771 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.382773 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.382773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.382959 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.383134 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:03.383135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:03.383136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:03.383203 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:03.383426 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:03.383427 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:03.383477 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 22:03:03.383539 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 22:03:03.383645 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 22:03:03.383783 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:03.377071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.9.112.124:38153)
0925 22:03:03.377142 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:03.377144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:03.377154 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:03.377231 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:03.377232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:03.377244 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:03.377302 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:03.377655 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.378149 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.378850 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:03.379044 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:03.379107 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:03.381148 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:03.381150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:03.381150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:03.381159 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:03.382671 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.383024 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.383025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.383026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.383080 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:03.383257 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:03.383257 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:03.383258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:03.383370 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:03.383428 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:03.383567 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 22:03:03.383691 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:03:03.385851 23026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3: Bootstrap starting.
I20260925 22:03:03.386708 23026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:03.387050 23026 log.cc:824] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:03.387804 23026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3: No bootstrap required, opened a new log
I20260925 22:03:03.389330 23026 raft_consensus.cc:348] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.389443 23026 raft_consensus.cc:374] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:03.389477 23026 raft_consensus.cc:729] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38652de7c9945838635f0d374c7e2d3, State: Initialized, Role: FOLLOWER
I20260925 22:03:03.389616 23026 consensus_queue.cc:260] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.389878 23217 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.389933 23026 sys_catalog.cc:564] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:03.395677 22958 master_runner.cc:263] No leader master found from master a38652de7c9945838635f0d374c7e2d3
I20260925 22:03:03.395732 22958 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:03:03.389990 23217 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:03:03.395921 23228 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:03:03.395963 23228 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:03:03.413125 23217 raft_consensus.cc:482] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:03.413259 23217 raft_consensus.cc:504] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.413520 23217 leader_election.cc:288] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81c153b5e61d4f358438810010e682d6 (127.9.112.125:43043), 2de4c74811934ef9ad459c662c4f20de (127.9.112.124:38153)
I20260925 22:03:03.413935 23150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a38652de7c9945838635f0d374c7e2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de4c74811934ef9ad459c662c4f20de" is_pre_election: true
I20260925 22:03:03.414094 23150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38652de7c9945838635f0d374c7e2d3 in term 0.
I20260925 22:03:03.414402 22976 leader_election.cc:302] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de4c74811934ef9ad459c662c4f20de, a38652de7c9945838635f0d374c7e2d3; no voters:
I20260925 22:03:03.414595 23217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:03:03.414645 23217 raft_consensus.cc:482] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:03:03.414667 23217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:03.415611 23217 raft_consensus.cc:504] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.415727 23217 leader_election.cc:288] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 81c153b5e61d4f358438810010e682d6 (127.9.112.125:43043), 2de4c74811934ef9ad459c662c4f20de (127.9.112.124:38153)
I20260925 22:03:03.416049 23150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a38652de7c9945838635f0d374c7e2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de4c74811934ef9ad459c662c4f20de"
I20260925 22:03:03.416129 23150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:03.417054 23150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38652de7c9945838635f0d374c7e2d3 in term 1.
I20260925 22:03:03.417281 22976 leader_election.cc:302] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de4c74811934ef9ad459c662c4f20de, a38652de7c9945838635f0d374c7e2d3; no voters:
I20260925 22:03:03.417495 23217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:03.417626 23217 raft_consensus.cc:686] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a38652de7c9945838635f0d374c7e2d3, State: Running, Role: LEADER
I20260925 22:03:03.417657 23075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a38652de7c9945838635f0d374c7e2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c153b5e61d4f358438810010e682d6" is_pre_election: true
I20260925 22:03:03.417718 23217 consensus_queue.cc:237] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } }
I20260925 22:03:03.417804 23075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38652de7c9945838635f0d374c7e2d3 in term 0.
I20260925 22:03:03.417927 23074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a38652de7c9945838635f0d374c7e2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c153b5e61d4f358438810010e682d6"
I20260925 22:03:03.417989 23074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:03.418149 23217 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a38652de7c9945838635f0d374c7e2d3. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.418212 23217 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:03.418579 23232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:03.418797 23232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:03.418927 23074 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38652de7c9945838635f0d374c7e2d3 in term 1.
I20260925 22:03:03.420924 23150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [term 1 FOLLOWER]: Refusing update from remote peer a38652de7c9945838635f0d374c7e2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:03.421301 23217 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:03.422655 23189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: SysCatalogTable state changed. Reason: New leader a38652de7c9945838635f0d374c7e2d3. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.422742 23189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:03:03.423249 23074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [term 1 FOLLOWER]: Refusing update from remote peer a38652de7c9945838635f0d374c7e2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:03.423622 23217 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:03.425207 23203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a38652de7c9945838635f0d374c7e2d3. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.425305 23203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:03:03.425981 23217 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.426077 23217 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:03.427413 23189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.427513 23189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:03:03.427855 23233 mvcc.cc:205] Tried to move back new op lower bound from 7333371016889626624 to 7333371016879534080. Current Snapshot: MvccSnapshot[applied={T|T < 7333371016889626624}]
I20260925 22:03:03.428462 23234 mvcc.cc:205] Tried to move back new op lower bound from 7333371016889626624 to 7333371016879534080. Current Snapshot: MvccSnapshot[applied={T|T < 7333371016889626624}]
I20260925 22:03:03.429100 23232 catalog_manager.cc:1347] Generated new cluster ID: 6cf92c014e96433d906033b456f474f0
I20260925 22:03:03.429147 23232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:03:03.429697 23217 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.429777 23217 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:03.430335 23203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a38652de7c9945838635f0d374c7e2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38652de7c9945838635f0d374c7e2d3" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42849 } } peers { permanent_uuid: "81c153b5e61d4f358438810010e682d6" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 43043 } } peers { permanent_uuid: "2de4c74811934ef9ad459c662c4f20de" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38153 } } }
I20260925 22:03:03.430409 23203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:03:03.430604 23236 mvcc.cc:205] Tried to move back new op lower bound from 7333371016889626624 to 7333371016879534080. Current Snapshot: MvccSnapshot[applied={T|T < 7333371016889626624}]
I20260925 22:03:03.450786 23232 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:03:03.451679 23232 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:03:03.468559 23232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a38652de7c9945838635f0d374c7e2d3: Generated new TSK 0
I20260925 22:03:03.468873 23232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:03:04.330545 23200 catalog_manager.cc:1259] Loaded cluster ID: 6cf92c014e96433d906033b456f474f0
I20260925 22:03:04.330665 23200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: loading cluster ID for follower catalog manager: success
I20260925 22:03:04.331458 23200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: acquiring CA information for follower catalog manager: success
I20260925 22:03:04.331715 23200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2de4c74811934ef9ad459c662c4f20de: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:03:04.367865 23214 catalog_manager.cc:1259] Loaded cluster ID: 6cf92c014e96433d906033b456f474f0
I20260925 22:03:04.367947 23214 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: loading cluster ID for follower catalog manager: success
I20260925 22:03:04.368831 23214 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: acquiring CA information for follower catalog manager: success
I20260925 22:03:04.369138 23214 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 81c153b5e61d4f358438810010e682d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:03:04.526495 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 22958
I20260925 22:03:04.532625 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23033
I20260925 22:03:04.540022 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23108
2026-09-25T22:03:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2852 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:03:04 dist-test-slave-794d krb5kdc[23249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:03:04 dist-test-slave-794d krb5kdc[23249](info): set up 2 sockets
Sep 25 22:03:04 dist-test-slave-794d krb5kdc[23249](info): commencing operation
krb5kdc: starting...
W20260925 22:03:05.516793 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:03:05 dist-test-slave-794d krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:03:05.641386 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35693
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:39567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:05.732929 23262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:05.733180 23262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:05.733203 23262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:05.734355 23262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:05.734391 23262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:05.734405 23262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:05.734416 23262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:05.734426 23262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:05.735914 23262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:39567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:05.736138 23262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:05.736368 23262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:05.741648 23270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:05.741636 23268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:05.741668 23262 server_base.cc:1034] running on GCE node
W20260925 22:03:05.741633 23267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:05.742128 23262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:05.742439 23262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:05.743599 23262 hybrid_clock.cc:648] HybridClock initialized: now 1790373785743577 us; error 34 us; skew 500 ppm
Sep 25 22:03:05 dist-test-slave-794d krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:05.746359 23262 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:05.746837 23262 webserver.cc:460] Webserver started at http://127.9.112.126:34793/ using document root <none> and password file <none>
I20260925 22:03:05.747051 23262 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:05.747092 23262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:05.747193 23262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:05.748389 23262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "6196fbce4cca42a4bb7024f54fc9d869"
format_stamp: "Formatted at 2026-09-25 22:03:05 on dist-test-slave-794d"
server_key: "dcbe4dfa337a0c6bf744aa624962df25"
server_key_iv: "7ca1407930dee535a1c6a67907d0704e"
server_key_version: "encryptionkey@0"
I20260925 22:03:05.748728 23262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "6196fbce4cca42a4bb7024f54fc9d869"
format_stamp: "Formatted at 2026-09-25 22:03:05 on dist-test-slave-794d"
server_key: "dcbe4dfa337a0c6bf744aa624962df25"
server_key_iv: "7ca1407930dee535a1c6a67907d0704e"
server_key_version: "encryptionkey@0"
I20260925 22:03:05.750067 23262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:05.750803 23277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:05.750972 23262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:03:05.751058 23262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "6196fbce4cca42a4bb7024f54fc9d869"
format_stamp: "Formatted at 2026-09-25 22:03:05 on dist-test-slave-794d"
server_key: "dcbe4dfa337a0c6bf744aa624962df25"
server_key_iv: "7ca1407930dee535a1c6a67907d0704e"
server_key_version: "encryptionkey@0"
I20260925 22:03:05.751119 23262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:05.772032 23262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:05.774212 23262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:05.774384 23262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:05.778789 23262 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35693
I20260925 22:03:05.778836 23329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35693 every 8 connection(s)
I20260925 22:03:05.779217 23262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:05.780220 23330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:05.782964 23330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869: Bootstrap starting.
I20260925 22:03:05.783629 23330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:05.783933 23330 log.cc:824] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:05.784758 23330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869: No bootstrap required, opened a new log
I20260925 22:03:05.786550 23330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6196fbce4cca42a4bb7024f54fc9d869" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35693 } }
I20260925 22:03:05.786718 23330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:05.786742 23330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6196fbce4cca42a4bb7024f54fc9d869, State: Initialized, Role: FOLLOWER
I20260925 22:03:05.786899 23330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6196fbce4cca42a4bb7024f54fc9d869" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35693 } }
I20260925 22:03:05.786980 23330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:03:05.787014 23330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:03:05.787066 23330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:05.787470 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23262
I20260925 22:03:05.787559 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:03:05.787703 23330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6196fbce4cca42a4bb7024f54fc9d869" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35693 } }
I20260925 22:03:05.787736 9665 external_mini_cluster.cc:1428] Setting key f69467d019502641dd6e80486348f50f
I20260925 22:03:05.787810 23330 leader_election.cc:302] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6196fbce4cca42a4bb7024f54fc9d869; no voters:
I20260925 22:03:05.788004 23330 leader_election.cc:288] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:03:05.788049 23333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:05.788225 23333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [term 1 LEADER]: Becoming Leader. State: Replica: 6196fbce4cca42a4bb7024f54fc9d869, State: Running, Role: LEADER
I20260925 22:03:05.788313 23330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:05.788630 23333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6196fbce4cca42a4bb7024f54fc9d869" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35693 } }
I20260925 22:03:05.789182 23334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6196fbce4cca42a4bb7024f54fc9d869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6196fbce4cca42a4bb7024f54fc9d869" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35693 } } }
I20260925 22:03:05.789279 23334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:05.789628 23335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6196fbce4cca42a4bb7024f54fc9d869. Latest consensus state: current_term: 1 leader_uuid: "6196fbce4cca42a4bb7024f54fc9d869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6196fbce4cca42a4bb7024f54fc9d869" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35693 } } }
I20260925 22:03:05.789695 23335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869 [sys.catalog]: This master's current role is: LEADER
W20260925 22:03:05.790560 23351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:03:05.790620 23351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:03:05.790699 23344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:05.790992 23344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:05.793853 23344 catalog_manager.cc:1347] Generated new cluster ID: dc4b81b9aa6348959d0bfb25d708b9cd
I20260925 22:03:05.793921 23344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:03:05 dist-test-slave-794d krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:05.799234 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:05.788448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.9.112.126:35693)
0925 22:03:05.788604 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:05.788615 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:05.788651 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:03:05.788794 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:05.788803 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:05.788865 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:05.789029 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:05.790011 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:05.791400 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:05.792543 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:05.793173 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:05.793309 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:05.795928 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:05.795933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:05.795933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:05.795948 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:05.797675 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:05.798017 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:05.798021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:05.798021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:05.798129 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:05.798317 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:05.798317 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:05.798318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:05.798590 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:05.798762 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:05.798876 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 22:03:05.799033 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260925 22:03:05.800276 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:03:05.812158 23344 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:03:05.812793 23344 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:03:05.821265 23344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6196fbce4cca42a4bb7024f54fc9d869: Generated new TSK 0
I20260925 22:03:05.821480 23344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:03:05.854707 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23262
2026-09-25T22:03:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1313 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:03:05 dist-test-slave-794d krb5kdc[23357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:03:05 dist-test-slave-794d krb5kdc[23357](info): set up 2 sockets
Sep 25 22:03:05 dist-test-slave-794d krb5kdc[23357](info): commencing operation
krb5kdc: starting...
W20260925 22:03:06.824705 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:03:06 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:03:06.957770 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38301
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:07.050153 23370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:07.050437 23370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:07.050462 23370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:07.052115 23370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:07.052196 23370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:07.052210 23370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:07.052222 23370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:07.052233 23370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:07.054173 23370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:07.054488 23370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:07.054728 23370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:07.061172 23378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.061386 23376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.061607 23370 server_base.cc:1034] running on GCE node
W20260925 22:03:07.061722 23375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.062157 23370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:07.062947 23370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:07.064179 23370 hybrid_clock.cc:648] HybridClock initialized: now 1790373787064167 us; error 45 us; skew 500 ppm
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:07.067762 23370 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:07.068406 23370 webserver.cc:460] Webserver started at http://127.9.112.126:42047/ using document root <none> and password file <none>
I20260925 22:03:07.068650 23370 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:07.068703 23370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:07.068845 23370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:07.070089 23370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "97111559da644e06ad7008d22c28f292"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "ad22cb7b9e0a5f97e049b98b48b4607b"
server_key_iv: "84e0d6cde3e544a2e473c80269b98f94"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.070470 23370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "97111559da644e06ad7008d22c28f292"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "ad22cb7b9e0a5f97e049b98b48b4607b"
server_key_iv: "84e0d6cde3e544a2e473c80269b98f94"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.071977 23370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:03:07.073082 23385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.073294 23370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:07.073428 23370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "97111559da644e06ad7008d22c28f292"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "ad22cb7b9e0a5f97e049b98b48b4607b"
server_key_iv: "84e0d6cde3e544a2e473c80269b98f94"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.073504 23370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:07.107621 23370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:07.110047 23370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:07.110211 23370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:07.115581 23370 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:38301
I20260925 22:03:07.115614 23437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:38301 every 8 connection(s)
I20260925 22:03:07.116029 23370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:07.117354 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:07.120296 23438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap starting.
I20260925 22:03:07.121074 23438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:07.121411 23438 log.cc:824] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:07.122282 23438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: No bootstrap required, opened a new log
I20260925 22:03:07.124166 23438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.124389 23438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:07.124428 23438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97111559da644e06ad7008d22c28f292, State: Initialized, Role: FOLLOWER
I20260925 22:03:07.124593 23438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.124683 23438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:03:07.124717 23438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:03:07.124773 23438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:07.124878 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23370
I20260925 22:03:07.125008 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:03:07.125205 9665 external_mini_cluster.cc:1428] Setting key 8708e151b42075bdca6393a1629e4a51
I20260925 22:03:07.126024 23438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.126158 23438 leader_election.cc:302] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97111559da644e06ad7008d22c28f292; no voters:
I20260925 22:03:07.126338 23438 leader_election.cc:288] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:03:07.126370 23443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:07.126480 23443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 LEADER]: Becoming Leader. State: Replica: 97111559da644e06ad7008d22c28f292, State: Running, Role: LEADER
I20260925 22:03:07.126650 23438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:07.126636 23443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.127432 23444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97111559da644e06ad7008d22c28f292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } } }
I20260925 22:03:07.127454 23445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97111559da644e06ad7008d22c28f292. Latest consensus state: current_term: 1 leader_uuid: "97111559da644e06ad7008d22c28f292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } } }
I20260925 22:03:07.127595 23445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:07.127789 23444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:07.127909 23451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:07.128626 23451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:07.130477 23451 catalog_manager.cc:1347] Generated new cluster ID: 864265067d94499295863ab57c23883b
I20260925 22:03:07.130532 23451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:07.138793 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.125901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.9.112.126:38301)
0925 22:03:07.126122 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.126133 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.126172 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:03:07.126318 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.126375 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.126449 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.126653 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:07.127407 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.128914 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.130023 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.130540 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.130687 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.133753 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:07.133759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:07.133759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:07.133775 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:07.136782 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.137235 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.137239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.137239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.137398 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.137587 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.137588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.137588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.137999 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:07.138185 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.138269 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.138530 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":114,"threads_started":1}
I20260925 22:03:07.159883 23451 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:03:07.160953 23451 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:03:07.167933 23451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Generated new TSK 0
I20260925 22:03:07.168200 23451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:03:07.201614 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38301
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:07.298959 23464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:07.299214 23464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:07.299240 23464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:07.300558 23464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:07.300606 23464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:07.300634 23464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:03:07.302279 23464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:07.302591 23464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:07.302873 23464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:07.308650 23470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.308696 23464 server_base.cc:1034] running on GCE node
W20260925 22:03:07.308655 23472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.308665 23469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.309226 23464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:07.309490 23464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:07.310638 23464 hybrid_clock.cc:648] HybridClock initialized: now 1790373787310626 us; error 33 us; skew 500 ppm
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:07.313881 23464 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:03:07.314460 23464 webserver.cc:460] Webserver started at http://127.9.112.65:37411/ using document root <none> and password file <none>
I20260925 22:03:07.314710 23464 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:07.314764 23464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:07.315016 23464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:07.316214 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "61172a1f9390fcfb75475046a33e8fb8"
server_key_iv: "aa8eeaa877783b24e02bad72601e7817"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.316642 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "61172a1f9390fcfb75475046a33e8fb8"
server_key_iv: "aa8eeaa877783b24e02bad72601e7817"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.318109 23464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:07.319089 23479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.319442 23464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:07.319547 23464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "61172a1f9390fcfb75475046a33e8fb8"
server_key_iv: "aa8eeaa877783b24e02bad72601e7817"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.319625 23464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:07.333268 23464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:07.336429 23464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:07.336627 23464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:07.336932 23464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:07.337306 23464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:07.337342 23464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.337378 23464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:07.337397 23464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.344760 23464 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:40827
I20260925 22:03:07.344825 23592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:40827 every 8 connection(s)
I20260925 22:03:07.345211 23464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:03:07.348544 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23464
I20260925 22:03:07.348646 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:03:07.348830 9665 external_mini_cluster.cc:1428] Setting key 4b3d0035b9bad6d15f6d7a6c8914a592
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:07.357685 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.346103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:43255 (local address 127.9.112.126:38301)
0925 22:03:07.346241 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.346244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.346498 (+ 254us) server_negotiation.cc:407] Connection header received
0925 22:03:07.347127 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.347129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.347151 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.347214 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:07.348457 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.349262 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.350671 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.350986 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.351070 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.354018 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:07.354019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:07.354020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:07.354029 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:07.355805 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.356450 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.356452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.356453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.356521 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.356868 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.356869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.356869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.357016 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:07.357073 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.357501 (+ 428us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.357599 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 22:03:07.357988 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.346026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:43255)
0925 22:03:07.346270 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.346373 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.347018 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.347307 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.347311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.347983 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:07.348237 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.348243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.349361 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.349362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.350553 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.350558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.351092 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.351094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.351230 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.351920 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:07.351932 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:07.353870 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:07.355905 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:07.355918 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:07.355922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:07.356282 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:07.356593 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:07.356594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:07.356595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:07.356786 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:07.357081 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:07.357090 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:07.357316 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.357457 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.357675 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
I20260925 22:03:07.358603 23593 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38301
I20260925 22:03:07.358721 23593 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:07.358985 23593 heartbeater.cc:511] Master 127.9.112.126:38301 requested a full tablet report, sending...
I20260925 22:03:07.359746 23402 ts_manager.cc:194] Registered new tserver with Master: db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827)
I20260925 22:03:07.360582 23402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:43255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:03:07.384577 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38301
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:07.479643 23598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:07.479893 23598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:07.479919 23598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:07.481279 23598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:07.481336 23598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:07.481367 23598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:03:07.483227 23598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:07.483481 23598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:07.483731 23598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:07.489543 23604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.489547 23603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.489636 23606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.489830 23598 server_base.cc:1034] running on GCE node
I20260925 22:03:07.490123 23598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:07.490383 23598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:07.491544 23598 hybrid_clock.cc:648] HybridClock initialized: now 1790373787491505 us; error 49 us; skew 500 ppm
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:07.494266 23598 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:03:07.494710 23598 webserver.cc:460] Webserver started at http://127.9.112.66:32987/ using document root <none> and password file <none>
I20260925 22:03:07.494928 23598 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:07.494977 23598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:07.495103 23598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:07.496204 23598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "0a95f8bcc71a2ba26e411becd9896eaa"
server_key_iv: "a5fda4b1b12eeda0dcfe7e21a655f4fd"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.496591 23598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "0a95f8bcc71a2ba26e411becd9896eaa"
server_key_iv: "a5fda4b1b12eeda0dcfe7e21a655f4fd"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.497977 23598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:07.498956 23613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.499265 23598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:07.499374 23598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "0a95f8bcc71a2ba26e411becd9896eaa"
server_key_iv: "a5fda4b1b12eeda0dcfe7e21a655f4fd"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.499452 23598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:07.514990 23598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:07.517198 23598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:07.517360 23598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:07.517598 23598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:07.517918 23598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:07.517972 23598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.518003 23598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:07.518028 23598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.523849 23598 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:45921
I20260925 22:03:07.523901 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:45921 every 8 connection(s)
I20260925 22:03:07.524287 23598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:07.531309 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23598
I20260925 22:03:07.531404 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:03:07.531653 9665 external_mini_cluster.cc:1428] Setting key 20bfd296ed300188446b31c6f3a34480
I20260925 22:03:07.534703 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.525052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:49243 (local address 127.9.112.126:38301)
0925 22:03:07.525110 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.525112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.525296 (+ 184us) server_negotiation.cc:407] Connection header received
0925 22:03:07.525816 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.525817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.525828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.525890 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:07.527099 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.527714 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.529020 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.529243 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.529308 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.531797 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:07.531799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:07.531799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:07.531810 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:07.533304 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.533743 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.533745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.533745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.533804 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.534038 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.534039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.534039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.534161 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:07.534219 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.534500 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.534595 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:03:07.534945 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.525002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:49243)
0925 22:03:07.525180 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.525223 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.525725 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.525973 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.525978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.526728 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:07.526969 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.526975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.527824 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.527825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.528901 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.528905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.529365 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.529366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.529508 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.530050 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:07.530059 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:07.531666 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:07.533406 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:07.533416 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:07.533419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:07.533668 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:07.533862 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:07.533863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:07.533863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:07.533982 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:07.534220 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:07.534228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:07.534379 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.534473 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.534650 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260925 22:03:07.535426 23727 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38301
I20260925 22:03:07.535552 23727 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:07.535761 23727 heartbeater.cc:511] Master 127.9.112.126:38301 requested a full tablet report, sending...
I20260925 22:03:07.536242 23402 ts_manager.cc:194] Registered new tserver with Master: ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921)
I20260925 22:03:07.536799 23402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:49243
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:03:07.559871 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38301
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:07.646011 23732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:07.646319 23732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:07.646363 23732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:07.648435 23732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:07.648535 23732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:07.648577 23732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:03:07.650492 23732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:07.650755 23732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:07.651000 23732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:07.656302 23737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.656296 23740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.656307 23738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.656519 23732 server_base.cc:1034] running on GCE node
I20260925 22:03:07.656893 23732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:07.657147 23732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:07.658329 23732 hybrid_clock.cc:648] HybridClock initialized: now 1790373787658314 us; error 28 us; skew 500 ppm
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:07.661084 23732 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:03:07.661553 23732 webserver.cc:460] Webserver started at http://127.9.112.67:35749/ using document root <none> and password file <none>
I20260925 22:03:07.661762 23732 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:07.661813 23732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:07.661952 23732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:07.663041 23732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "761296f7277e48b194096ed5816a149b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "03b49c74df86a8b19c3fc8eed89aee3a"
server_key_iv: "2d51411957b307cdfecccdd6130993cb"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.663393 23732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "761296f7277e48b194096ed5816a149b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "03b49c74df86a8b19c3fc8eed89aee3a"
server_key_iv: "2d51411957b307cdfecccdd6130993cb"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.664891 23732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:03:07.665642 23747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.665782 23732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:03:07.665868 23732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "761296f7277e48b194096ed5816a149b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "03b49c74df86a8b19c3fc8eed89aee3a"
server_key_iv: "2d51411957b307cdfecccdd6130993cb"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.665931 23732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:07.680222 23732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:07.682469 23732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:07.682623 23732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:07.682874 23732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:07.683175 23732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:07.683209 23732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.683241 23732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:07.683259 23732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.689153 23732 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:40559
I20260925 22:03:07.689244 23860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:40559 every 8 connection(s)
I20260925 22:03:07.689523 23732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:03:07.695262 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23732
I20260925 22:03:07.695340 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:03:07.695485 9665 external_mini_cluster.cc:1428] Setting key 299eb65ef5ac829bb615e2c4f2b0c410
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:07.698679 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.690248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:56773 (local address 127.9.112.126:38301)
0925 22:03:07.690313 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.690316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.690467 (+ 151us) server_negotiation.cc:407] Connection header received
0925 22:03:07.691016 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.691017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.691028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.691077 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:07.692007 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.692581 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.693650 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.693856 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.693918 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.696243 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:07.696245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:07.696245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:07.696253 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:07.697515 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.697881 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.697882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.697882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.697933 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.698147 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.698148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.698148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.698248 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:07.698291 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.698510 (+ 219us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.698571 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:03:07.698814 23863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.690197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:56773)
0925 22:03:07.690353 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.690390 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.690894 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.691135 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.691138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.691703 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:07.691899 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.691903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.692696 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.692697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.693501 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.693503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.693941 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.693942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.694058 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.694629 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:07.694638 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:07.696151 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:07.697588 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:07.697597 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:07.697599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:07.697821 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:07.697980 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:07.697980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:07.697980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:07.698074 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:07.698290 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:07.698294 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:07.698414 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.698492 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.698617 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260925 22:03:07.699237 23861 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38301
I20260925 22:03:07.699329 23861 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:07.699504 23861 heartbeater.cc:511] Master 127.9.112.126:38301 requested a full tablet report, sending...
I20260925 22:03:07.699905 23402 ts_manager.cc:194] Registered new tserver with Master: 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
I20260925 22:03:07.700492 23402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:56773
I20260925 22:03:07.707460 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:03:07.715415 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.709012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.9.112.126:38301)
0925 22:03:07.709104 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.709106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.709200 (+ 94us) server_negotiation.cc:407] Connection header received
0925 22:03:07.709285 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.709286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.709308 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.709362 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:07.710024 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.710549 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.711325 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.711530 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.711594 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.712928 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:07.712929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:07.712930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:07.712938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:07.714273 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.714624 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.714626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.714626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.714680 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:07.714840 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:07.714841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:07.714842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:07.715037 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:07.715129 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.715244 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.715321 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 22:03:07.717324 23402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:03:07.719070 23402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:03:07.725828 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.721657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45921 (local address 127.0.0.1:48712)
0925 22:03:07.721951 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.721963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.722029 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.722312 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.722315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.722318 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:07.722399 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.722400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.723458 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.723459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.724681 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.724684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.725503 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.725504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.725597 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.725611 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.725676 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.725735 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":83,"threads_started":1}
I20260925 22:03:07.725900 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.722052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.9.112.65:40827)
0925 22:03:07.722233 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.722237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.722246 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:03:07.722289 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.722291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.722305 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.722524 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:07.722663 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.723833 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.724913 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.725576 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.725649 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.725700 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.725771 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.725812 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260925 22:03:07.726085 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.721811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48712 (local address 127.9.112.66:45921)
0925 22:03:07.722093 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.722098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.722110 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:03:07.722163 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.722165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.722192 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.722472 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:07.722549 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.723368 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.724782 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.725428 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.725489 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.725554 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.725968 (+ 414us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.726016 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":75,"threads_started":1}
I20260925 22:03:07.726431 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.721828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40827 (local address 127.0.0.1:34928)
0925 22:03:07.721964 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.721972 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.722029 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.722471 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.722472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.722474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:07.722570 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.722572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.723927 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.723928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.724837 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.724839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.725647 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.725647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.725702 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.725705 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.725747 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.725779 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260925 22:03:07.726724 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.721752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.9.112.67:40559)
0925 22:03:07.722166 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.722171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.722181 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:07.722218 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.722220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.722237 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.722310 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:07.722651 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.723798 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.725147 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.725872 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.725941 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.725997 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.726428 (+ 431us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.726649 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":83,"threads_started":1}
I20260925 22:03:07.726563 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.721625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40559 (local address 127.0.0.1:35568)
0925 22:03:07.721797 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.721838 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.721900 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.722406 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.722407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.722410 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:07.722540 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.722543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.723899 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.723901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.725061 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.725063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.726032 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.726032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.726105 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.726110 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.726394 (+ 284us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.726444 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":9,"thread_start_us":64,"threads_started":1}
I20260925 22:03:07.726904 23661 tablet_service.cc:1467] Processing CreateTablet for tablet 751233ab94e9401db76da56e77d014d7 (DEFAULT_TABLE table=test-table [id=91226e038cda41ddb1f6719568fd3ac5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:07.727326 23527 tablet_service.cc:1467] Processing CreateTablet for tablet 751233ab94e9401db76da56e77d014d7 (DEFAULT_TABLE table=test-table [id=91226e038cda41ddb1f6719568fd3ac5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:07.727520 23661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751233ab94e9401db76da56e77d014d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:07.727752 23527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751233ab94e9401db76da56e77d014d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:07.727686 23795 tablet_service.cc:1467] Processing CreateTablet for tablet 751233ab94e9401db76da56e77d014d7 (DEFAULT_TABLE table=test-table [id=91226e038cda41ddb1f6719568fd3ac5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:07.727984 23795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751233ab94e9401db76da56e77d014d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:07.730361 23878 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Bootstrap starting.
I20260925 22:03:07.730960 23878 tablet_bootstrap.cc:654] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:07.731101 23879 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap starting.
I20260925 22:03:07.731582 23878 log.cc:824] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:07.731714 23879 tablet_bootstrap.cc:654] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:07.732215 23879 log.cc:824] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:07.732457 23878 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: No bootstrap required, opened a new log
I20260925 22:03:07.732533 23878 ts_tablet_manager.cc:1397] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:07.732604 23880 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Bootstrap starting.
I20260925 22:03:07.733107 23879 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: No bootstrap required, opened a new log
I20260925 22:03:07.733173 23879 ts_tablet_manager.cc:1397] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:03:07.733259 23880 tablet_bootstrap.cc:654] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:07.733758 23880 log.cc:824] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:07.734730 23880 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: No bootstrap required, opened a new log
I20260925 22:03:07.734858 23880 ts_tablet_manager.cc:1397] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:07.734745 23878 raft_consensus.cc:348] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.734926 23878 raft_consensus.cc:374] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:07.734993 23878 raft_consensus.cc:729] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef913e46e1f74076a42701b7f4c1e5b8, State: Initialized, Role: FOLLOWER
I20260925 22:03:07.734942 23879 raft_consensus.cc:348] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.735134 23879 raft_consensus.cc:374] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:07.735163 23878 consensus_queue.cc:260] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.735203 23879 raft_consensus.cc:729] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1ea375728b40bfa7bcf8c2ccd9cb5b, State: Initialized, Role: FOLLOWER
I20260925 22:03:07.735353 23879 consensus_queue.cc:260] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.735513 23727 heartbeater.cc:503] Master 127.9.112.126:38301 was elected leader, sending a full tablet report...
I20260925 22:03:07.735703 23593 heartbeater.cc:503] Master 127.9.112.126:38301 was elected leader, sending a full tablet report...
I20260925 22:03:07.735502 23878 ts_tablet_manager.cc:1428] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:03:07.735706 23879 ts_tablet_manager.cc:1428] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:07.736946 23880 raft_consensus.cc:348] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.737082 23880 raft_consensus.cc:374] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:07.737102 23880 raft_consensus.cc:729] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 761296f7277e48b194096ed5816a149b, State: Initialized, Role: FOLLOWER
I20260925 22:03:07.737208 23880 consensus_queue.cc:260] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.737443 23880 ts_tablet_manager.cc:1428] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:03:07.737457 23861 heartbeater.cc:503] Master 127.9.112.126:38301 was elected leader, sending a full tablet report...
I20260925 22:03:07.743682 23885 raft_consensus.cc:482] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:07.743821 23885 raft_consensus.cc:504] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.744232 23885 leader_election.cc:288] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
I20260925 22:03:07.747102 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.744540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:48097 (local address 127.9.112.66:45921)
0925 22:03:07.744615 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.744617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.744626 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:03:07.744681 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.744682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.744691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.744745 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:07.744889 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.745364 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.746304 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.746750 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.746815 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.746825 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.746991 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.747035 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:03:07.747102 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.744446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45921 (local address 127.9.112.65:48097)
0925 22:03:07.744570 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.744582 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.744627 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.744758 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.744759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.744760 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:07.744835 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.744836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.745443 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.745444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.746218 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.746220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.746825 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.746826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.746914 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.746917 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.746963 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.747019 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":25}
I20260925 22:03:07.747157 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.744480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:54239 (local address 127.9.112.67:40559)
0925 22:03:07.744587 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.744590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.744720 (+ 130us) server_negotiation.cc:407] Connection header received
0925 22:03:07.744779 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.744780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.744789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.744840 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:07.745001 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.745448 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.746399 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.746833 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.746896 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.746905 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.747050 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.747092 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 22:03:07.747262 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.744497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40559 (local address 127.9.112.65:54239)
0925 22:03:07.744653 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.744662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:07.744720 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:07.744843 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:07.744844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:07.744845 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:07.744911 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.744912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.745567 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.745568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.746302 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:07.746303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.746903 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.746904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:07.746990 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.746994 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:07.747038 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:03:07.747079 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260925 22:03:07.747593 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751233ab94e9401db76da56e77d014d7" candidate_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "761296f7277e48b194096ed5816a149b" is_pre_election: true
I20260925 22:03:07.747748 23815 raft_consensus.cc:2454] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db1ea375728b40bfa7bcf8c2ccd9cb5b in term 0.
I20260925 22:03:07.747922 23681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751233ab94e9401db76da56e77d014d7" candidate_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" is_pre_election: true
I20260925 22:03:07.747987 23480 leader_election.cc:302] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761296f7277e48b194096ed5816a149b, db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters:
I20260925 22:03:07.748090 23681 raft_consensus.cc:2454] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db1ea375728b40bfa7bcf8c2ccd9cb5b in term 0.
I20260925 22:03:07.748109 23885 raft_consensus.cc:2781] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:03:07.748186 23885 raft_consensus.cc:482] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:03:07.748208 23885 raft_consensus.cc:3037] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:07.748984 23885 raft_consensus.cc:504] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.749105 23885 leader_election.cc:288] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 1 election: Requested vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
I20260925 22:03:07.749306 23681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751233ab94e9401db76da56e77d014d7" candidate_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
I20260925 22:03:07.749318 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751233ab94e9401db76da56e77d014d7" candidate_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "761296f7277e48b194096ed5816a149b"
I20260925 22:03:07.749388 23815 raft_consensus.cc:3037] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:07.749388 23681 raft_consensus.cc:3037] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:07.750089 23815 raft_consensus.cc:2454] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db1ea375728b40bfa7bcf8c2ccd9cb5b in term 1.
I20260925 22:03:07.750275 23681 raft_consensus.cc:2454] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db1ea375728b40bfa7bcf8c2ccd9cb5b in term 1.
I20260925 22:03:07.750257 23480 leader_election.cc:302] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761296f7277e48b194096ed5816a149b, db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters:
I20260925 22:03:07.750383 23885 raft_consensus.cc:2781] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:07.750559 23885 raft_consensus.cc:686] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 LEADER]: Becoming Leader. State: Replica: db1ea375728b40bfa7bcf8c2ccd9cb5b, State: Running, Role: LEADER
I20260925 22:03:07.750654 23885 consensus_queue.cc:237] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:07.751551 23401 catalog_manager.cc:5495] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b reported cstate change: term changed from 0 to 1, leader changed from <none> to db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65). New cstate: current_term: 1 leader_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } health_report { overall_health: HEALTHY } } }
I20260925 22:03:07.763882 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.761328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.9.112.65:40827)
0925 22:03:07.761429 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:07.761431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:07.761440 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:03:07.761477 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:07.761478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:07.761495 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:07.761630 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:07.761781 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.762259 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.763025 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:07.763230 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:07.763286 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:07.763460 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:07.763611 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:03:07.763675 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:07.763746 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 22:03:07.763765 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 22:03:07.765877 23815 raft_consensus.cc:1261] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Refusing update from remote peer db1ea375728b40bfa7bcf8c2ccd9cb5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:07.765928 23681 raft_consensus.cc:1261] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Refusing update from remote peer db1ea375728b40bfa7bcf8c2ccd9cb5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:07.766260 23885 consensus_queue.cc:1035] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:07.766405 23888 consensus_queue.cc:1035] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:07.768906 23891 mvcc.cc:205] Tried to move back new op lower bound from 7333371034686328832 to 7333371034627383296. Current Snapshot: MvccSnapshot[applied={T|T < 7333371034686328832}]
I20260925 22:03:07.769580 23892 mvcc.cc:205] Tried to move back new op lower bound from 7333371034686328832 to 7333371034627383296. Current Snapshot: MvccSnapshot[applied={T|T < 7333371034686328832}]
I20260925 22:03:07.770047 23893 mvcc.cc:205] Tried to move back new op lower bound from 7333371034686328832 to 7333371034627383296. Current Snapshot: MvccSnapshot[applied={T|T < 7333371034686328832}]
I20260925 22:03:07.771253 23402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:07.771342 23402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:07.772133 23402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Sending DeleteTablet for 3 replicas of tablet 751233ab94e9401db76da56e77d014d7
I20260925 22:03:07.772519 23527 tablet_service.cc:1514] Processing DeleteTablet for tablet 751233ab94e9401db76da56e77d014d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:07 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:34928
I20260925 22:03:07.772554 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23464
I20260925 22:03:07.772627 23661 tablet_service.cc:1514] Processing DeleteTablet for tablet 751233ab94e9401db76da56e77d014d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:07 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:48712
I20260925 22:03:07.773041 23795 tablet_service.cc:1514] Processing DeleteTablet for tablet 751233ab94e9401db76da56e77d014d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:07 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:35568
I20260925 22:03:07.773260 23901 tablet_replica.cc:330] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: stopping tablet replica
I20260925 22:03:07.773360 23901 raft_consensus.cc:2229] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:07.773464 23901 raft_consensus.cc:2258] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:07.773672 23900 tablet_replica.cc:330] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: stopping tablet replica
I20260925 22:03:07.773777 23900 raft_consensus.cc:2229] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:07.773881 23900 raft_consensus.cc:2258] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:07.774015 23901 ts_tablet_manager.cc:1905] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:07.774288 23900 ts_tablet_manager.cc:1905] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:07.775928 23901 ts_tablet_manager.cc:1918] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:07.775974 23900 ts_tablet_manager.cc:1918] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:07.776021 23901 log.cc:1196] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/751233ab94e9401db76da56e77d014d7
I20260925 22:03:07.776028 23900 log.cc:1196] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/751233ab94e9401db76da56e77d014d7
I20260925 22:03:07.776290 23901 ts_tablet_manager.cc:1939] T 751233ab94e9401db76da56e77d014d7 P 761296f7277e48b194096ed5816a149b: Deleting consensus metadata
I20260925 22:03:07.776290 23900 ts_tablet_manager.cc:1939] T 751233ab94e9401db76da56e77d014d7 P ef913e46e1f74076a42701b7f4c1e5b8: Deleting consensus metadata
I20260925 22:03:07.776777 23386 catalog_manager.cc:4841] TS 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): tablet 751233ab94e9401db76da56e77d014d7 (table test-table [id=91226e038cda41ddb1f6719568fd3ac5]) successfully deleted
I20260925 22:03:07.776896 23387 catalog_manager.cc:4841] TS ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): tablet 751233ab94e9401db76da56e77d014d7 (table test-table [id=91226e038cda41ddb1f6719568fd3ac5]) successfully deleted
W20260925 22:03:07.778224 23386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:40827 (error 108)
I20260925 22:03:07.778497 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23598
I20260925 22:03:07.779126 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:07.778688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40827 (local address 127.0.0.1:34938)
0925 22:03:07.779054 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:07.779093 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
W20260925 22:03:07.780509 23386 catalog_manager.cc:4584] TS db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 751233ab94e9401db76da56e77d014d7: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40827: connect: Connection refused (error 111)
I20260925 22:03:07.783070 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23732
I20260925 22:03:07.787227 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23370
I20260925 22:03:07.792297 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38301
--webserver_interface=127.9.112.126
--webserver_port=42047
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:07.875036 23903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:07.875277 23903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:07.875309 23903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:07.876585 23903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:07.876633 23903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:07.876650 23903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:07.876667 23903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:07.876680 23903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:07.878281 23903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=42047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:07.878566 23903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:07.878788 23903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:07.887718 23910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:07.887750 23912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.887861 23903 server_base.cc:1034] running on GCE node
W20260925 22:03:07.887727 23909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:07.888278 23903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:07.888682 23903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:07.889842 23903 hybrid_clock.cc:648] HybridClock initialized: now 1790373787889807 us; error 58 us; skew 500 ppm
Sep 25 22:03:07 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:07.892484 23903 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:07.892994 23903 webserver.cc:460] Webserver started at http://127.9.112.126:42047/ using document root <none> and password file <none>
I20260925 22:03:07.893220 23903 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:07.893285 23903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:07.894817 23903 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:07.895650 23919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:07.895860 23903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:07.895955 23903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "97111559da644e06ad7008d22c28f292"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "ad22cb7b9e0a5f97e049b98b48b4607b"
server_key_iv: "84e0d6cde3e544a2e473c80269b98f94"
server_key_version: "encryptionkey@0"
I20260925 22:03:07.896425 23903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:07.916838 23903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:07.919116 23903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:07.919263 23903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:07.923882 23903 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:38301
I20260925 22:03:07.923956 23971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:38301 every 8 connection(s)
I20260925 22:03:07.924417 23903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:07.926119 23972 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:03:07.926829 23972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap starting.
I20260925 22:03:07.927727 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23903
I20260925 22:03:07.927997 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:40827
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:38301
--webserver_port=37411
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:07.937086 23972 log.cc:824] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:07.939913 23972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:03:07.940254 23972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap complete.
I20260925 22:03:07.943475 23972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.943819 23972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97111559da644e06ad7008d22c28f292, State: Initialized, Role: FOLLOWER
I20260925 22:03:07.943986 23972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.944049 23972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:03:07.944098 23972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:03:07.944154 23972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:03:07.945626 23972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.945760 23972 leader_election.cc:302] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97111559da644e06ad7008d22c28f292; no voters:
I20260925 22:03:07.945936 23972 leader_election.cc:288] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:03:07.946061 23976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:03:07.946211 23972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:07.946277 23976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 LEADER]: Becoming Leader. State: Replica: 97111559da644e06ad7008d22c28f292, State: Running, Role: LEADER
I20260925 22:03:07.946427 23976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:07.946792 23976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97111559da644e06ad7008d22c28f292. Latest consensus state: current_term: 2 leader_uuid: "97111559da644e06ad7008d22c28f292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } } }
I20260925 22:03:07.946765 23977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "97111559da644e06ad7008d22c28f292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } } }
I20260925 22:03:07.947160 23977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:07.947080 23976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:07.947748 23988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:07.948830 23988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:07.948967 23988 catalog_manager.cc:1259] Loaded cluster ID: 864265067d94499295863ab57c23883b
I20260925 22:03:07.949007 23988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:03:07.949314 23988 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 22:03:07.954172 23988 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 22:03:07.957486 23988 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:03:07.957755 23988 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Loaded TSK: 0
I20260925 22:03:07.957996 23988 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:03:08.020849 23974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.021080 23974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.021107 23974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.022423 23974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.022487 23974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.022516 23974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:03:08.024374 23974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:40827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=37411
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.024657 23974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.024884 23974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:08.030071 23999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.030035 24000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.030284 24002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.030155 23974 server_base.cc:1034] running on GCE node
I20260925 22:03:08.030588 23974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.030850 23974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.032054 23974 hybrid_clock.cc:648] HybridClock initialized: now 1790373788032037 us; error 33 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.035342 23974 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:03:08.035892 23974 webserver.cc:460] Webserver started at http://127.9.112.65:37411/ using document root <none> and password file <none>
I20260925 22:03:08.036146 23974 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.036221 23974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.037761 23974 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:08.038573 24009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.038760 23974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:08.038841 23974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "61172a1f9390fcfb75475046a33e8fb8"
server_key_iv: "aa8eeaa877783b24e02bad72601e7817"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.039189 23974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.057339 23974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:08.059615 23974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:08.059732 23974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:08.059971 23974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:08.060508 24016 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:03:08.062616 23974 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:03:08.062690 23974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:03:08.062733 23974 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:03:08.063402 23974 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:03:08.063441 23974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:03:08.063544 24016 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap starting.
I20260925 22:03:08.070309 23974 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:40827
I20260925 22:03:08.070375 24123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:40827 every 8 connection(s)
I20260925 22:03:08.070776 23974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:03:08.074174 24016 log.cc:824] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:08.074235 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 23974
I20260925 22:03:08.074497 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:45921
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:38301
--webserver_port=32987
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:08.077016 24016 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:03:08.077705 24016 tablet_bootstrap.cc:492] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap complete.
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:08.078799 24016 ts_tablet_manager.cc:1397] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260925 22:03:08.080008 24016 raft_consensus.cc:348] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.080322 24016 raft_consensus.cc:729] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1ea375728b40bfa7bcf8c2ccd9cb5b, State: Initialized, Role: FOLLOWER
I20260925 22:03:08.080534 24016 consensus_queue.cc:260] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.080787 24016 ts_tablet_manager.cc:1428] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 22:03:08.082214 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.071488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:58539)
0925 22:03:08.071870 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.071904 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.072450 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.072867 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.072871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.073167 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:08.073352 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.073357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.074706 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.074707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.075896 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.075899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.076828 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.076830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.076969 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.077612 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:08.077620 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:08.078972 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:08.080701 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.080708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.080710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.080964 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.081206 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.081207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.081207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.081307 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.081636 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:08.081642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:08.081766 (+ 124us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.081849 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.082018 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":111,"threads_started":1}
I20260925 22:03:08.082214 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.071575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:58539 (local address 127.9.112.126:38301)
0925 22:03:08.071793 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.071802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.072022 (+ 220us) server_negotiation.cc:407] Connection header received
0925 22:03:08.072664 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.072669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.072714 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.072834 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:08.073476 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.074576 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.075996 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.076619 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.076717 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.079068 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:08.079072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:08.079072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:08.079084 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:08.080635 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.081042 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.081045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.081046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.081131 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.081362 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.081363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.081363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.081574 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:08.081645 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.081907 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.082022 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260925 22:03:08.083098 24124 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38301
I20260925 22:03:08.083186 24124 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:08.083341 24124 heartbeater.cc:511] Master 127.9.112.126:38301 requested a full tablet report, sending...
I20260925 22:03:08.084012 23936 ts_manager.cc:194] Registered new tserver with Master: db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827)
I20260925 22:03:08.085132 23936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:58539
I20260925 22:03:08.086047 24124 heartbeater.cc:503] Master 127.9.112.126:38301 was elected leader, sending a full tablet report...
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:03:08.093127 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.084581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40827 (local address 127.0.0.1:34952)
0925 22:03:08.084954 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.084989 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.085061 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.085351 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.085356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.086282 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:08.086341 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.086344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.087224 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.087226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.087798 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.087800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.088241 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.088242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.088321 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.088952 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:08.088957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:08.090232 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:08.091819 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.091827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.091829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.092060 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.092276 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.092276 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.092277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.092365 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.092734 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:08.092743 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:08.092800 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.092876 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.093016 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":90,"threads_started":1}
I20260925 22:03:08.093127 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.084819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.9.112.65:40827)
0925 22:03:08.085177 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.085182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.085191 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:03:08.085222 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.085224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.085237 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.085294 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:08.086424 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.087143 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.087880 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.088173 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.088231 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.090314 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:08.090322 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:08.090322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:08.090330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:08.091732 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.092140 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.092143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.092144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.092218 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.092414 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.092415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.092415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.092670 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:08.092748 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.092918 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.093016 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":60,"threads_started":1}
I20260925 22:03:08.093628 24058 tablet_service.cc:1514] Processing DeleteTablet for tablet 751233ab94e9401db76da56e77d014d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:07 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:34952
I20260925 22:03:08.094146 24133 tablet_replica.cc:330] stopping tablet replica
I20260925 22:03:08.094269 24133 raft_consensus.cc:2229] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:08.094339 24133 raft_consensus.cc:2258] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:08.094671 24133 ts_tablet_manager.cc:1905] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:08.097421 24133 ts_tablet_manager.cc:1918] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:08.097517 24133 log.cc:1196] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/751233ab94e9401db76da56e77d014d7
I20260925 22:03:08.097729 24133 ts_tablet_manager.cc:1939] T 751233ab94e9401db76da56e77d014d7 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Deleting consensus metadata
I20260925 22:03:08.098344 23920 catalog_manager.cc:4841] TS db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827): tablet 751233ab94e9401db76da56e77d014d7 (table test-table [id=91226e038cda41ddb1f6719568fd3ac5]) successfully deleted
W20260925 22:03:08.164052 24129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.164278 24129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.164304 24129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.165637 24129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.165699 24129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.165726 24129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:03:08.167348 24129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:45921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=32987
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.167699 24129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.167937 24129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:08.173326 24139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.173329 24138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.173331 24141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.173732 24129 server_base.cc:1034] running on GCE node
I20260925 22:03:08.173950 24129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.174206 24129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.175392 24129 hybrid_clock.cc:648] HybridClock initialized: now 1790373788175374 us; error 31 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.177999 24129 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:03:08.178431 24129 webserver.cc:460] Webserver started at http://127.9.112.66:32987/ using document root <none> and password file <none>
I20260925 22:03:08.178676 24129 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.178743 24129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.180065 24129 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.180687 24148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.180840 24129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.180922 24129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "0a95f8bcc71a2ba26e411becd9896eaa"
server_key_iv: "a5fda4b1b12eeda0dcfe7e21a655f4fd"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.181281 24129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.201547 24129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:08.203600 24129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:08.203749 24129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:08.203971 24129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:08.204275 24129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:08.204319 24129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.204377 24129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:08.204402 24129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.210110 24129 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:45921
I20260925 22:03:08.210173 24261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:45921 every 8 connection(s)
I20260925 22:03:08.210508 24129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:03:08.214600 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24129
I20260925 22:03:08.214905 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:40559
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:38301
--webserver_port=35749
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:08.220683 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.211340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:35483 (local address 127.9.112.126:38301)
0925 22:03:08.211401 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.211403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.211547 (+ 144us) server_negotiation.cc:407] Connection header received
0925 22:03:08.212040 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.212041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.212052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.212117 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:08.213117 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.213633 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.214871 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.215111 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.215179 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.217674 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:08.217676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:08.217676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:08.217683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:08.218971 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.219469 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.219470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.219470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.219526 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.219899 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.219901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.219901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.220046 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:08.220112 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.220523 (+ 411us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.220611 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:03:08.220917 24264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.211234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:35483)
0925 22:03:08.211391 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.211435 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.211913 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.212176 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.212179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.212806 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:08.213003 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.213008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.213784 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.213785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.214759 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.214763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.215202 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.215203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.215348 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.215967 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:08.215978 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:08.217563 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:08.219056 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.219068 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.219071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.219393 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.219645 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.219645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.219646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.219795 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.220125 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:08.220132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:08.220322 (+ 190us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.220472 (+ 150us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.220664 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260925 22:03:08.221465 24262 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38301
I20260925 22:03:08.221580 24262 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:08.221782 24262 heartbeater.cc:511] Master 127.9.112.126:38301 requested a full tablet report, sending...
I20260925 22:03:08.222170 23936 ts_manager.cc:194] Registered new tserver with Master: ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921)
I20260925 22:03:08.222536 23936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:35483
W20260925 22:03:08.298502 24265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.298729 24265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.298754 24265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.300019 24265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.300071 24265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.300099 24265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:03:08.301767 24265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:40559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=35749
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.302003 24265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.302232 24265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:08.307492 24273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.307569 24270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.307590 24271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.307694 24265 server_base.cc:1034] running on GCE node
I20260925 22:03:08.307916 24265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.308225 24265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.309398 24265 hybrid_clock.cc:648] HybridClock initialized: now 1790373788309362 us; error 70 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.312052 24265 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:03:08.312568 24265 webserver.cc:460] Webserver started at http://127.9.112.67:35749/ using document root <none> and password file <none>
I20260925 22:03:08.312798 24265 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.312880 24265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.314309 24265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.314926 24280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.315140 24265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.315217 24265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "761296f7277e48b194096ed5816a149b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "03b49c74df86a8b19c3fc8eed89aee3a"
server_key_iv: "2d51411957b307cdfecccdd6130993cb"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.315558 24265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.322203 24265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:08.324103 24265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:08.324236 24265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:08.324502 24265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:08.324810 24265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:08.324842 24265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.324877 24265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:08.324901 24265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.330979 24265 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:40559
I20260925 22:03:08.331039 24393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:40559 every 8 connection(s)
I20260925 22:03:08.331403 24265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:08.340440 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24265
I20260925 22:03:08.340921 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.332171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:54831 (local address 127.9.112.126:38301)
0925 22:03:08.332251 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.332253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.332423 (+ 170us) server_negotiation.cc:407] Connection header received
0925 22:03:08.333358 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.333361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.333372 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.333440 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:08.334209 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.334721 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.335793 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.336048 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.336114 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.338334 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:08.338336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:08.338337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:08.338345 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:08.339485 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.339869 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.339870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.339871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.339925 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.340142 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.340143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.340144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.340282 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:08.340368 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.340738 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.340833 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 22:03:08.341123 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.332123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:54831)
0925 22:03:08.332279 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.332322 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.333240 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.333624 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.333629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.333902 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:08.334112 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.334116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.334820 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.334822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.335689 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.335692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.336143 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.336144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.336257 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.336843 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:08.336852 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:08.338235 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:08.339567 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.339574 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.339576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.339799 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.339981 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:08.339982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:08.339982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:08.340083 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:08.340365 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:08.340372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:08.340509 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.340694 (+ 185us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.340881 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260925 22:03:08.341614 24394 heartbeater.cc:347] Connected to a master server at 127.9.112.126:38301
I20260925 22:03:08.341715 24394 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:08.341885 24394 heartbeater.cc:511] Master 127.9.112.126:38301 requested a full tablet report, sending...
I20260925 22:03:08.342263 23936 ts_manager.cc:194] Registered new tserver with Master: 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
I20260925 22:03:08.342723 23936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:54831
I20260925 22:03:08.347384 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.340846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.9.112.126:38301)
0925 22:03:08.341074 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.341077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.341083 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:03:08.341102 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.341102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.341112 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.341155 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:08.341880 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.342546 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.343580 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.343771 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.343842 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.344985 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:08.344986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:08.344986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:08.344992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:08.346284 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.346665 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.346666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.346666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.346716 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:08.346962 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:08.346963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:08.346963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:08.347057 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:08.347180 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.347255 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.347316 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":52,"threads_started":1}
I20260925 22:03:08.348206 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:03:08.351460 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.348575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51336 (local address 127.9.112.126:38301)
0925 22:03:08.348617 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.348619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.348649 (+ 30us) server_negotiation.cc:407] Connection header received
0925 22:03:08.348722 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.348723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.348759 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.348812 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:08.348931 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.349486 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.350261 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.350563 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.350859 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.351081 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:08.351199 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:03:08.351309 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.351353 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.351374 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:03:08.351807 23936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:03:08.353587 23936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:03:08.356545 24058 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7fa2017a2747fa9144969baff31fc1 (DEFAULT_TABLE table=test-table [id=c4bf7d58f2ec4c1fbee2d6b9e597a259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:08.356791 24058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7fa2017a2747fa9144969baff31fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:08.358278 24016 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap starting.
I20260925 22:03:08.358572 24016 tablet_bootstrap.cc:654] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:08.359375 24016 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: No bootstrap required, opened a new log
I20260925 22:03:08.359419 24016 ts_tablet_manager.cc:1397] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:08.359557 24016 raft_consensus.cc:348] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.359618 24016 raft_consensus.cc:374] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:08.359673 24016 raft_consensus.cc:729] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1ea375728b40bfa7bcf8c2ccd9cb5b, State: Initialized, Role: FOLLOWER
I20260925 22:03:08.359724 24016 consensus_queue.cc:260] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.359822 24016 ts_tablet_manager.cc:1428] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.360131 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.356021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45921 (local address 127.0.0.1:48728)
0925 22:03:08.356079 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.356091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.356130 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.356579 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.356580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.356581 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:08.356635 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.356637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.357853 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.357853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.358982 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.358986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.359820 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.359821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.359914 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.359943 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.360000 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.360054 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":23}
I20260925 22:03:08.360452 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.356171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.9.112.66:45921)
0925 22:03:08.356364 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.356370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.356380 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:08.356423 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.356425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.356443 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.356559 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:08.356721 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.357752 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.359071 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.359733 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.359804 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.359872 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.360300 (+ 428us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.360374 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260925 22:03:08.361193 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.356021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40559 (local address 127.0.0.1:35572)
0925 22:03:08.356376 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.356388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.356451 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.356695 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.356696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.356698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:08.356743 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.356744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.357747 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.357748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.359144 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.359146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.360936 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.360937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.361007 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.361011 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.361067 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.361111 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":22,"thread_start_us":138,"threads_started":1}
I20260925 22:03:08.361253 24401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.356152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.9.112.67:40559)
0925 22:03:08.356440 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.356443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.356453 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:08.356516 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.356519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.356532 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.356613 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:08.356811 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.357643 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.359230 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.360859 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.360918 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.360957 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.361082 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.361124 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":92,"threads_started":1}
I20260925 22:03:08.361238 24196 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7fa2017a2747fa9144969baff31fc1 (DEFAULT_TABLE table=test-table [id=c4bf7d58f2ec4c1fbee2d6b9e597a259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:08.361596 24196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7fa2017a2747fa9144969baff31fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:08.361896 24328 tablet_service.cc:1467] Processing CreateTablet for tablet 6c7fa2017a2747fa9144969baff31fc1 (DEFAULT_TABLE table=test-table [id=c4bf7d58f2ec4c1fbee2d6b9e597a259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:08.362341 24328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c7fa2017a2747fa9144969baff31fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:08.365520 24404 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Bootstrap starting.
I20260925 22:03:08.365927 24405 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Bootstrap starting.
I20260925 22:03:08.366230 24404 tablet_bootstrap.cc:654] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:08.366606 24405 tablet_bootstrap.cc:654] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:08.366752 24404 log.cc:824] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:08.367134 24405 log.cc:824] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:08.367555 24404 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: No bootstrap required, opened a new log
I20260925 22:03:08.367625 24404 ts_tablet_manager.cc:1397] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:08.368072 24405 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: No bootstrap required, opened a new log
I20260925 22:03:08.368139 24405 ts_tablet_manager.cc:1397] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:08.369649 24404 raft_consensus.cc:348] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.369853 24404 raft_consensus.cc:374] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:08.369887 24404 raft_consensus.cc:729] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef913e46e1f74076a42701b7f4c1e5b8, State: Initialized, Role: FOLLOWER
I20260925 22:03:08.370036 24404 consensus_queue.cc:260] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.370028 24405 raft_consensus.cc:348] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.370188 24405 raft_consensus.cc:374] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:08.370229 24405 raft_consensus.cc:729] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 761296f7277e48b194096ed5816a149b, State: Initialized, Role: FOLLOWER
I20260925 22:03:08.370344 24404 ts_tablet_manager.cc:1428] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:03:08.370414 24262 heartbeater.cc:503] Master 127.9.112.126:38301 was elected leader, sending a full tablet report...
I20260925 22:03:08.370398 24405 consensus_queue.cc:260] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.370669 24405 ts_tablet_manager.cc:1428] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:03:08.370697 24394 heartbeater.cc:503] Master 127.9.112.126:38301 was elected leader, sending a full tablet report...
I20260925 22:03:08.380738 24409 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:08.380856 24409 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.381400 24409 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827)
I20260925 22:03:08.384418 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.381734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:39349 (local address 127.9.112.65:40827)
0925 22:03:08.381785 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.381787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.381794 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:03:08.381817 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.381818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.381838 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.381894 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:08.382035 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.382542 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.383515 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.384020 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.384088 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.384173 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.384276 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.384318 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:03:08.384421 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.381683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:42547 (local address 127.9.112.66:45921)
0925 22:03:08.381725 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.381727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.381842 (+ 115us) server_negotiation.cc:407] Connection header received
0925 22:03:08.381878 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.381880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.381889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.381934 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:08.382107 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.382576 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.383521 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.384000 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.384063 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.384073 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.384290 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.384333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:03:08.384430 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.381574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40827 (local address 127.9.112.67:39349)
0925 22:03:08.381646 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.381658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.381701 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.381904 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.381905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.381907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:08.381975 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.381977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.382613 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.382614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.383419 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.383421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.384123 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.384124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.384207 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.384211 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.384263 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.384313 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":6}
I20260925 22:03:08.384451 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.381592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45921 (local address 127.9.112.67:42547)
0925 22:03:08.381767 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.381776 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.381833 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.381942 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:08.381943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:08.381944 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:08.382044 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.382046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.382646 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.382647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.383429 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:08.383432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.384130 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.384131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:08.384218 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.384222 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:08.384274 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:03:08.384317 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":5,"thread_start_us":104,"threads_started":1}
I20260925 22:03:08.384739 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7fa2017a2747fa9144969baff31fc1" candidate_uuid: "761296f7277e48b194096ed5816a149b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" is_pre_election: true
I20260925 22:03:08.384922 24078 raft_consensus.cc:2454] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 761296f7277e48b194096ed5816a149b in term 0.
I20260925 22:03:08.384922 24216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7fa2017a2747fa9144969baff31fc1" candidate_uuid: "761296f7277e48b194096ed5816a149b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" is_pre_election: true
I20260925 22:03:08.385053 24216 raft_consensus.cc:2454] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 761296f7277e48b194096ed5816a149b in term 0.
I20260925 22:03:08.385188 24281 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761296f7277e48b194096ed5816a149b, db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters:
I20260925 22:03:08.385327 24409 raft_consensus.cc:2781] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:03:08.385385 24409 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:03:08.385417 24409 raft_consensus.cc:3037] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:08.386112 24409 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.386238 24409 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [CANDIDATE]: Term 1 election: Requested vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827)
I20260925 22:03:08.386408 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7fa2017a2747fa9144969baff31fc1" candidate_uuid: "761296f7277e48b194096ed5816a149b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b"
I20260925 22:03:08.386502 24078 raft_consensus.cc:3037] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:08.386483 24216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c7fa2017a2747fa9144969baff31fc1" candidate_uuid: "761296f7277e48b194096ed5816a149b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
I20260925 22:03:08.386662 24216 raft_consensus.cc:3037] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:08.387207 24078 raft_consensus.cc:2454] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 761296f7277e48b194096ed5816a149b in term 1.
I20260925 22:03:08.387367 24281 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761296f7277e48b194096ed5816a149b, db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters:
I20260925 22:03:08.387418 24216 raft_consensus.cc:2454] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 761296f7277e48b194096ed5816a149b in term 1.
I20260925 22:03:08.387467 24409 raft_consensus.cc:2781] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:08.387621 24409 raft_consensus.cc:686] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 1 LEADER]: Becoming Leader. State: Replica: 761296f7277e48b194096ed5816a149b, State: Running, Role: LEADER
I20260925 22:03:08.387718 24409 consensus_queue.cc:237] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.388577 23935 catalog_manager.cc:5495] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b reported cstate change: term changed from 0 to 1, leader changed from <none> to 761296f7277e48b194096ed5816a149b (127.9.112.67). New cstate: current_term: 1 leader_uuid: "761296f7277e48b194096ed5816a149b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } health_report { overall_health: UNKNOWN } } }
I20260925 22:03:08.396714 24401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.394269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.9.112.67:40559)
0925 22:03:08.394326 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:08.394328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:08.394334 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:03:08.394365 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:08.394366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:08.394382 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:08.394440 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:08.394602 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.395088 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.395805 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:08.395997 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:08.396051 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:08.396214 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:08.396408 (+ 194us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:03:08.396509 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:08.396532 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 22:03:08.396616 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:03:08.398746 24216 raft_consensus.cc:1261] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Refusing update from remote peer 761296f7277e48b194096ed5816a149b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:08.398886 24078 raft_consensus.cc:1261] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Refusing update from remote peer 761296f7277e48b194096ed5816a149b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:08.399107 24409 consensus_queue.cc:1035] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:08.399262 24411 consensus_queue.cc:1035] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:08.402328 24417 mvcc.cc:205] Tried to move back new op lower bound from 7333371037278150656 to 7333371037236768768. Current Snapshot: MvccSnapshot[applied={T|T < 7333371037278150656}]
I20260925 22:03:08.403801 23935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:08.403914 23935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:08.404748 23935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Sending DeleteTablet for 3 replicas of tablet 6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:08.405102 24328 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7fa2017a2747fa9144969baff31fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:08 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:35572
I20260925 22:03:08.405124 24058 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7fa2017a2747fa9144969baff31fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:08 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:34952
I20260925 22:03:08.405221 24133 tablet_replica.cc:330] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: stopping tablet replica
I20260925 22:03:08.405239 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23974
I20260925 22:03:08.405263 24133 raft_consensus.cc:2229] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:08.405318 24133 raft_consensus.cc:2258] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:08.405344 24196 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c7fa2017a2747fa9144969baff31fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:08 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:48728
I20260925 22:03:08.405596 24423 tablet_replica.cc:330] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: stopping tablet replica
I20260925 22:03:08.405685 24423 raft_consensus.cc:2229] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:03:08.405776 24423 raft_consensus.cc:2258] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:08.405918 24424 tablet_replica.cc:330] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: stopping tablet replica
I20260925 22:03:08.406011 24424 raft_consensus.cc:2229] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:08.406117 24424 raft_consensus.cc:2258] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:08.406270 24423 ts_tablet_manager.cc:1905] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:08.406592 24424 ts_tablet_manager.cc:1905] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:08.408414 24423 ts_tablet_manager.cc:1918] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:08.408507 24423 log.cc:1196] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:08.408504 24424 ts_tablet_manager.cc:1918] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:08.408605 24424 log.cc:1196] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:08.408802 24423 ts_tablet_manager.cc:1939] T 6c7fa2017a2747fa9144969baff31fc1 P 761296f7277e48b194096ed5816a149b: Deleting consensus metadata
I20260925 22:03:08.408896 24424 ts_tablet_manager.cc:1939] T 6c7fa2017a2747fa9144969baff31fc1 P ef913e46e1f74076a42701b7f4c1e5b8: Deleting consensus metadata
I20260925 22:03:08.409212 23920 catalog_manager.cc:4841] TS 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): tablet 6c7fa2017a2747fa9144969baff31fc1 (table test-table [id=c4bf7d58f2ec4c1fbee2d6b9e597a259]) successfully deleted
I20260925 22:03:08.409296 23921 catalog_manager.cc:4841] TS ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): tablet 6c7fa2017a2747fa9144969baff31fc1 (table test-table [id=c4bf7d58f2ec4c1fbee2d6b9e597a259]) successfully deleted
W20260925 22:03:08.412123 23920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:40827 (error 108)
I20260925 22:03:08.412518 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 24129
I20260925 22:03:08.414077 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.413948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40827 (local address 127.0.0.1:34956)
0925 22:03:08.414001 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.414037 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:03:08.414208 23920 catalog_manager.cc:4584] TS db1ea375728b40bfa7bcf8c2ccd9cb5b (127.9.112.65:40827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c7fa2017a2747fa9144969baff31fc1: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40827: connect: Connection refused (error 111)
I20260925 22:03:08.417726 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 24265
I20260925 22:03:08.422331 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 23903
I20260925 22:03:08.426980 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38301
--webserver_interface=127.9.112.126
--webserver_port=42047
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:08.513077 24426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.513334 24426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.513360 24426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.514622 24426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:08.514670 24426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.514683 24426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.514694 24426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:08.514705 24426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:08.516381 24426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=42047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.516678 24426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.516949 24426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:08.523164 24434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.523197 24431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.523244 24432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.523658 24426 server_base.cc:1034] running on GCE node
I20260925 22:03:08.523975 24426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.524394 24426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.525616 24426 hybrid_clock.cc:648] HybridClock initialized: now 1790373788525575 us; error 73 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.529089 24426 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:08.529778 24426 webserver.cc:460] Webserver started at http://127.9.112.126:42047/ using document root <none> and password file <none>
I20260925 22:03:08.530047 24426 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.530097 24426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.531814 24426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.532678 24441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.532923 24426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.533036 24426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "97111559da644e06ad7008d22c28f292"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "ad22cb7b9e0a5f97e049b98b48b4607b"
server_key_iv: "84e0d6cde3e544a2e473c80269b98f94"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.533538 24426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.550475 24426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:08.553278 24426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:08.553440 24426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:08.558712 24426 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:38301
I20260925 22:03:08.558765 24493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:38301 every 8 connection(s)
I20260925 22:03:08.559194 24426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:08.561092 24494 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:03:08.561856 24494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap starting.
I20260925 22:03:08.562584 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24426
I20260925 22:03:08.562909 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:40827
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:38301
--webserver_port=37411
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:08.574487 24494 log.cc:824] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:08.578342 24494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:03:08.578821 24494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292: Bootstrap complete.
I20260925 22:03:08.583249 24494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:08.583710 24494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97111559da644e06ad7008d22c28f292, State: Initialized, Role: FOLLOWER
I20260925 22:03:08.583925 24494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:08.584002 24494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:03:08.584043 24494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:03:08.584086 24494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 2 FOLLOWER]: Advancing to term 3
I20260925 22:03:08.586072 24494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:08.586506 24494 leader_election.cc:302] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97111559da644e06ad7008d22c28f292; no voters:
I20260925 22:03:08.586861 24494 leader_election.cc:288] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 22:03:08.587015 24498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 22:03:08.587216 24498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [term 3 LEADER]: Becoming Leader. State: Replica: 97111559da644e06ad7008d22c28f292, State: Running, Role: LEADER
I20260925 22:03:08.587268 24494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:08.587345 24498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } }
I20260925 22:03:08.587709 24499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "97111559da644e06ad7008d22c28f292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } } }
I20260925 22:03:08.587797 24499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:08.587975 24500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97111559da644e06ad7008d22c28f292. Latest consensus state: current_term: 3 leader_uuid: "97111559da644e06ad7008d22c28f292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97111559da644e06ad7008d22c28f292" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38301 } } }
I20260925 22:03:08.588022 24500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97111559da644e06ad7008d22c28f292 [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:08.588219 24505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:08.589352 24505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:08.589462 24505 catalog_manager.cc:1259] Loaded cluster ID: 864265067d94499295863ab57c23883b
I20260925 22:03:08.589493 24505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 22:03:08.589794 24505 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdcbfcf29ad google::LogMessage::Fail() at ??:0
@ 0x7fdcbfcf588a google::LogMessage::SendToLog() at ??:0
@ 0x7fdcbfcf24de google::LogMessage::Flush() at ??:0
@ 0x7fdcbfcf3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdcc7c170f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdcc7c208b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdcc1bc00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdcc1bb5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdcc2a196db start_thread at ??:0
@ 0x7fdcbe7f061f clone at ??:0
W20260925 22:03:08.676419 24496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.676690 24496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.676730 24496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.678038 24496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.678081 24496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.678121 24496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:03:08.679769 24496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:40827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=37411
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.680068 24496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.680327 24496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:03:08.686228 24496 server_base.cc:1034] running on GCE node
W20260925 22:03:08.686226 24530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.686426 24533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.686264 24531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.687112 24496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.687419 24496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.688591 24496 hybrid_clock.cc:648] HybridClock initialized: now 1790373788688578 us; error 33 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.691377 24496 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:03:08.691910 24496 webserver.cc:460] Webserver started at http://127.9.112.65:37411/ using document root <none> and password file <none>
I20260925 22:03:08.692142 24496 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.692191 24496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.693827 24496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.694612 24540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.694813 24496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.694921 24496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "61172a1f9390fcfb75475046a33e8fb8"
server_key_iv: "aa8eeaa877783b24e02bad72601e7817"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.695392 24496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.715900 24496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:08.718528 24496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:08.718693 24496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:08.718983 24496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:08.719486 24547 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:03:08.721799 24496 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:03:08.721865 24496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260925 22:03:08.721917 24496 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:03:08.722693 24496 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:03:08.722729 24496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:03:08.722846 24547 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap starting.
I20260925 22:03:08.730931 24496 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:40827
I20260925 22:03:08.730976 24654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:40827 every 8 connection(s)
I20260925 22:03:08.731658 24496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:03:08.736848 24547 log.cc:824] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:08.740545 24547 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:03:08.741003 24547 tablet_bootstrap.cc:492] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Bootstrap complete.
I20260925 22:03:08.741035 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24496
I20260925 22:03:08.741325 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:45921
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:38301
--webserver_port=32987
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:08.742398 24547 ts_tablet_manager.cc:1397] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.008s
I20260925 22:03:08.744882 24547 raft_consensus.cc:348] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.745639 24547 raft_consensus.cc:729] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1ea375728b40bfa7bcf8c2ccd9cb5b, State: Initialized, Role: FOLLOWER
I20260925 22:03:08.745903 24547 consensus_queue.cc:260] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:08.746294 24547 ts_tablet_manager.cc:1428] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b: Time spent starting tablet: real 0.004s user 0.005s sys 0.001s
W20260925 22:03:08.811836 24657 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:03:08.732607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:60251)
0925 22:03:08.732806 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.732857 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:08.733416 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:08.811577 (+ 78161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
W20260925 22:03:08.812105 24655 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:38301 (0 consecutive failures): Network error: Failed to ping master at 127.9.112.126:38301: Client connection negotiation failed: client connection to 127.9.112.126:38301: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:03:08.812472 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.812364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:60267)
0925 22:03:08.812406 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.812450 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 22:03:08.812911 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.812750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:37877)
0925 22:03:08.812842 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.812883 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260925 22:03:08.813043 24655 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:03:08.839998 24659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.840266 24659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.840293 24659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.841616 24659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.841668 24659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.841696 24659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:03:08.843287 24659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:45921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=32987
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.843535 24659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.843765 24659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:08.849435 24665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.849472 24666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.849391 24668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.849612 24659 server_base.cc:1034] running on GCE node
I20260925 22:03:08.849831 24659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.850072 24659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.851270 24659 hybrid_clock.cc:648] HybridClock initialized: now 1790373788851255 us; error 33 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.854336 24659 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:03:08.854853 24659 webserver.cc:460] Webserver started at http://127.9.112.66:32987/ using document root <none> and password file <none>
I20260925 22:03:08.855110 24659 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.855163 24659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.856863 24659 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:03:08.857893 24675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.858101 24659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:08.858208 24659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "ef913e46e1f74076a42701b7f4c1e5b8"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "0a95f8bcc71a2ba26e411becd9896eaa"
server_key_iv: "a5fda4b1b12eeda0dcfe7e21a655f4fd"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.858707 24659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.866578 24659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:08.869196 24659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:08.869405 24659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:08.869714 24659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:08.870146 24659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:08.870191 24659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.870235 24659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:08.870250 24659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.877264 24659 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:45921
I20260925 22:03:08.877341 24788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:45921 every 8 connection(s)
I20260925 22:03:08.877709 24659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:03:08.878474 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24659
I20260925 22:03:08.878726 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:40559
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:38301
--webserver_port=35749
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:08.878943 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.878467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:38787)
0925 22:03:08.878645 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.878734 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
W20260925 22:03:08.879348 24789 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:38301 (0 consecutive failures): Network error: Failed to ping master at 127.9.112.126:38301: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
I20260925 22:03:08.879637 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.879521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:49645)
0925 22:03:08.879573 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.879610 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 22:03:08.880041 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:08.879943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:43517)
0925 22:03:08.879990 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:08.880019 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 22:03:08.880159 24789 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:03:08.971362 24792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:08.971640 24792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:08.971678 24792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:08.973028 24792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:08.973079 24792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:08.973119 24792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:03:08.974731 24792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:40559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=35749
--tserver_master_addrs=127.9.112.126:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:08.975003 24792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:08.975262 24792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:08.980841 24800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.980835 24797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:08.980839 24798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:08.981106 24792 server_base.cc:1034] running on GCE node
I20260925 22:03:08.981362 24792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:08.981611 24792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:08.982780 24792 hybrid_clock.cc:648] HybridClock initialized: now 1790373788982765 us; error 32 us; skew 500 ppm
Sep 25 22:03:08 dist-test-slave-794d krb5kdc[23357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:08.985684 24792 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:03:08.986227 24792 webserver.cc:460] Webserver started at http://127.9.112.67:35749/ using document root <none> and password file <none>
I20260925 22:03:08.986438 24792 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:08.986482 24792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:08.987996 24792 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.988732 24807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:08.988907 24792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:08.989001 24792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "761296f7277e48b194096ed5816a149b"
format_stamp: "Formatted at 2026-09-25 22:03:07 on dist-test-slave-794d"
server_key: "03b49c74df86a8b19c3fc8eed89aee3a"
server_key_iv: "2d51411957b307cdfecccdd6130993cb"
server_key_version: "encryptionkey@0"
I20260925 22:03:08.989456 24792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:08.999538 24792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:09.002135 24792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:09.002307 24792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:09.002616 24792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:09.002969 24792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:09.003005 24792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:09.003037 24792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:09.003060 24792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:09.010339 24792 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:40559
I20260925 22:03:09.010416 24920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:40559 every 8 connection(s)
I20260925 22:03:09.010809 24792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:03:09.012027 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:09.011574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:59971)
0925 22:03:09.011768 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:09.011857 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
W20260925 22:03:09.012529 24921 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:38301 (0 consecutive failures): Network error: Failed to ping master at 127.9.112.126:38301: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
I20260925 22:03:09.012862 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:09.012768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:42331)
0925 22:03:09.012811 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:09.012840 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 22:03:09.013124 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:09.013074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:54755)
0925 22:03:09.013094 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:09.013112 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260925 22:03:09.013206 24921 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:03:09.015300 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24792
I20260925 22:03:09.813862 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:09.813521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:56201)
0925 22:03:09.813712 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:09.813779 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260925 22:03:09.880985 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:09.880619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:47713)
0925 22:03:09.880818 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:09.880904 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260925 22:03:10.004962 24927 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:10.005136 24927 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:10.005653 24927 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
I20260925 22:03:10.013846 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.013554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:41905)
0925 22:03:10.013741 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:10.013799 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
Sep 25 22:03:10 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.67@KRBTEST.COM
Sep 25 22:03:10 dist-test-slave-794d krb5kdc[23357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.66@KRBTEST.COM
I20260925 22:03:10.022856 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.005931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40559 (local address 127.9.112.65:49313)
0925 22:03:10.006087 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:10.006100 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:10.006167 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:10.007147 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:10.007152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:10.007750 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:10.008016 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:10.008024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.009417 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.009418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:10.010359 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:10.010363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.011055 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.011056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:10.011206 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:10.015993 (+ 4787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:10.016006 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:10.017896 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:10.021016 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021016 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:10.021017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:10.021148 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:10.021484 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021484 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:10.021485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:10.021544 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:10.022148 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:10.022149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:10.022360 (+ 211us) client_negotiation.cc:769] Sending connection context
0925 22:03:10.022509 (+ 149us) client_negotiation.cc:240] Negotiation successful
0925 22:03:10.022727 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":22,"thread_start_us":66,"threads_started":1}
I20260925 22:03:10.022953 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.006032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:55903 (local address 127.9.112.66:45921)
0925 22:03:10.006221 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:10.006236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:10.006272 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:03:10.006839 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:10.006842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:10.006895 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:10.007035 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:10.008162 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.009258 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.010498 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.011232 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.011766 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:10.018340 (+ 6574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:10.018345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:10.018346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:10.018360 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:10.020494 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021049 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:10.021053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:10.021053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:10.021219 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021653 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:10.021654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:10.021654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:10.022049 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:10.022149 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:10.022638 (+ 489us) server_negotiation.cc:299] Negotiation successful
0925 22:03:10.022818 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":59,"threads_started":1}
I20260925 22:03:10.022990 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.006054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:49313 (local address 127.9.112.67:40559)
0925 22:03:10.006288 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:10.006300 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:10.006330 (+ 30us) server_negotiation.cc:407] Connection header received
0925 22:03:10.006852 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:10.006855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:10.006928 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:10.007128 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:10.008195 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.009302 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.010454 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.010963 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.011079 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:10.018087 (+ 7008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:10.018095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:10.018095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:10.018114 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:10.020497 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021279 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:10.021282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:10.021282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:10.021392 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021652 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:10.021653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:10.021653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:10.022072 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:10.022160 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:10.022634 (+ 474us) server_negotiation.cc:299] Negotiation successful
0925 22:03:10.022824 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260925 22:03:10.022934 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.005874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45921 (local address 127.9.112.65:55903)
0925 22:03:10.005976 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:10.005990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:10.006060 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:10.007148 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:10.007153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:10.007759 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:10.008016 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:10.008026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.009381 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.009383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:10.010366 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:10.010371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:10.011345 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:10.011347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:10.011416 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:10.015992 (+ 4576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:10.016005 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:10.018210 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:10.020628 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:10.020637 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:10.020640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:10.020933 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:10.021351 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:10.021352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:10.021352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:10.021505 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:10.022137 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:10.022145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:10.022359 (+ 214us) client_negotiation.cc:769] Sending connection context
0925 22:03:10.022501 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 22:03:10.022826 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":231}
W20260925 22:03:10.024389 24541 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
W20260925 22:03:10.024528 24542 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:10.024590 24542 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters: 761296f7277e48b194096ed5816a149b, ef913e46e1f74076a42701b7f4c1e5b8
I20260925 22:03:10.024717 24927 raft_consensus.cc:2726] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:03:10.814801 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.814537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:52227)
0925 22:03:10.814699 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:10.814750 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260925 22:03:10.882066 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:10.881701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:55517)
0925 22:03:10.881941 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:10.882007 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":128,"threads_started":1}
I20260925 22:03:11.014741 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:11.014499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:47001)
0925 22:03:11.014661 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:11.014690 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260925 22:03:11.815927 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:11.815456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:57681)
0925 22:03:11.815777 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:11.815855 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":185,"threads_started":1}
I20260925 22:03:11.883072 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:11.882725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:37839)
0925 22:03:11.882936 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:11.883005 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260925 22:03:12.016142 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:12.015840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:36419)
0925 22:03:12.016045 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:12.016075 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":119,"threads_started":1}
I20260925 22:03:12.233440 24938 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:12.233590 24938 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:12.233801 24938 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
W20260925 22:03:12.234427 24541 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
W20260925 22:03:12.234541 24542 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:12.234581 24542 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters: 761296f7277e48b194096ed5816a149b, ef913e46e1f74076a42701b7f4c1e5b8
I20260925 22:03:12.234681 24938 raft_consensus.cc:2726] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:03:12.816928 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:12.816581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:43943)
0925 22:03:12.816793 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:12.816863 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260925 22:03:12.884166 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:12.883834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:54107)
0925 22:03:12.884042 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:12.884113 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":112,"threads_started":1}
I20260925 22:03:13.016983 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:13.016625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:44921)
0925 22:03:13.016831 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:13.016900 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
I20260925 22:03:13.817957 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:13.817565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:35035)
0925 22:03:13.817783 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:13.817871 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
I20260925 22:03:13.885341 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:13.884944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:56985)
0925 22:03:13.885168 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:13.885272 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":114,"threads_started":1}
I20260925 22:03:13.951924 24944 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:13.952046 24944 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:13.952303 24944 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
W20260925 22:03:13.952883 24541 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
W20260925 22:03:13.953071 24542 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:13.953101 24542 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters: 761296f7277e48b194096ed5816a149b, ef913e46e1f74076a42701b7f4c1e5b8
I20260925 22:03:13.953189 24944 raft_consensus.cc:2726] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:03:14.018010 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:14.017643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:56083)
0925 22:03:14.017860 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:14.017938 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260925 22:03:14.818994 24946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:14.818678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:51153)
0925 22:03:14.818879 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:14.818945 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":93,"threads_started":1}
I20260925 22:03:14.886489 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:14.886106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:47421)
0925 22:03:14.886377 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:14.886439 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":155,"threads_started":1}
I20260925 22:03:15.019029 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:15.018712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:33143)
0925 22:03:15.018898 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:15.018949 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260925 22:03:15.819896 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:15.819573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:52015)
0925 22:03:15.819781 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:15.819845 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260925 22:03:15.887547 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:15.887226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:46625)
0925 22:03:15.887416 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:15.887481 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":83,"threads_started":1}
I20260925 22:03:16.020066 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:16.019727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:58091)
0925 22:03:16.019951 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:16.020016 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":104,"threads_started":1}
I20260925 22:03:16.471853 24952 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:16.471994 24952 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:16.472251 24952 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
W20260925 22:03:16.472815 24542 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
W20260925 22:03:16.472971 24541 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:16.473019 24541 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters: 761296f7277e48b194096ed5816a149b, ef913e46e1f74076a42701b7f4c1e5b8
I20260925 22:03:16.473117 24952 raft_consensus.cc:2726] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:03:16.820904 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:16.820587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:56549)
0925 22:03:16.820792 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:16.820852 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260925 22:03:16.888620 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:16.888259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:59143)
0925 22:03:16.888484 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:16.888554 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260925 22:03:17.021018 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:17.020716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:42215)
0925 22:03:17.020890 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:17.020951 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260925 22:03:17.821931 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:17.821614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:55247)
0925 22:03:17.821805 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:17.821871 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260925 22:03:17.889741 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:17.889223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:45567)
0925 22:03:17.889511 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:17.889654 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260925 22:03:18.022004 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:18.021687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:51119)
0925 22:03:18.021890 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:18.021952 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":107,"threads_started":1}
I20260925 22:03:18.822973 24959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:18.822667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:41437)
0925 22:03:18.822860 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:18.822924 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260925 22:03:18.890661 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:18.890338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:51461)
0925 22:03:18.890531 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:18.890594 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260925 22:03:19.023057 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:19.022719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:60539)
0925 22:03:19.022948 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:19.023012 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":121,"threads_started":1}
I20260925 22:03:19.824003 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:19.823704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:50095)
0925 22:03:19.823866 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:19.823937 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260925 22:03:19.891609 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:19.891232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:57801)
0925 22:03:19.891455 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:19.891557 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":105,"threads_started":1}
I20260925 22:03:20.024178 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:20.023795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:33153)
0925 22:03:20.024030 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:20.024099 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":126,"threads_started":1}
I20260925 22:03:20.825134 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:20.824792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:37317)
0925 22:03:20.825007 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:20.825063 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260925 22:03:20.893013 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:20.892631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:39781)
0925 22:03:20.892866 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:20.892927 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260925 22:03:21.025104 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:21.024795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:55209)
0925 22:03:21.024992 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:21.025037 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260925 22:03:21.826390 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:21.826028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:59123)
0925 22:03:21.826258 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:21.826325 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":135,"threads_started":1}
I20260925 22:03:21.876472 24969 raft_consensus.cc:482] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:21.876641 24969 raft_consensus.cc:504] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef913e46e1f74076a42701b7f4c1e5b8" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45921 } } peers { permanent_uuid: "761296f7277e48b194096ed5816a149b" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40559 } } peers { permanent_uuid: "db1ea375728b40bfa7bcf8c2ccd9cb5b" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40827 } }
I20260925 22:03:21.876860 24969 leader_election.cc:288] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921), 761296f7277e48b194096ed5816a149b (127.9.112.67:40559)
W20260925 22:03:21.877449 24541 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 761296f7277e48b194096ed5816a149b (127.9.112.67:40559): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
W20260925 22:03:21.877592 24542 leader_election.cc:341] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef913e46e1f74076a42701b7f4c1e5b8 (127.9.112.66:45921): Not found: Tablet not found: 6c7fa2017a2747fa9144969baff31fc1
I20260925 22:03:21.877723 24542 leader_election.cc:302] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db1ea375728b40bfa7bcf8c2ccd9cb5b; no voters: 761296f7277e48b194096ed5816a149b, ef913e46e1f74076a42701b7f4c1e5b8
I20260925 22:03:21.877816 24969 raft_consensus.cc:2726] T 6c7fa2017a2747fa9144969baff31fc1 P db1ea375728b40bfa7bcf8c2ccd9cb5b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:03:21.894059 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:21.893747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:54867)
0925 22:03:21.893925 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:21.894007 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":101,"threads_started":1}
I20260925 22:03:22.026067 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:22.025770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:59749)
0925 22:03:22.025968 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:22.026018 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":83,"threads_started":1}
I20260925 22:03:22.827416 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:22.827035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:57631)
0925 22:03:22.827170 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:22.827209 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260925 22:03:22.895100 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:22.894741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:38765)
0925 22:03:22.894976 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:22.895044 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":120,"threads_started":1}
I20260925 22:03:23.027184 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:23.026771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:41655)
0925 22:03:23.027069 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:23.027134 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":97,"threads_started":1}
I20260925 22:03:23.828614 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:23.828213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.65:60075)
0925 22:03:23.828502 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:23.828564 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":92,"threads_started":1}
I20260925 22:03:23.896164 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:23.895826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.66:39029)
0925 22:03:23.896030 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:23.896100 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
I20260925 22:03:24.019697 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 24496
I20260925 22:03:24.025285 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 24659
I20260925 22:03:24.028447 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:24.028102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:38301 (local address 127.9.112.67:36793)
0925 22:03:24.028313 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:24.028383 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":132,"threads_started":1}
I20260925 22:03:24.031543 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 24792
2026-09-25T22:03:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18179 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:03:24 dist-test-slave-794d krb5kdc[24980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:03:24 dist-test-slave-794d krb5kdc[24980](info): set up 2 sockets
Sep 25 22:03:24 dist-test-slave-794d krb5kdc[24980](info): commencing operation
krb5kdc: starting...
W20260925 22:03:25.036650 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:03:25.176069 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40127
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:25.273151 24993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:25.273437 24993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:25.273464 24993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:25.274843 24993 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 22:03:25.274900 24993 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 22:03:25.274914 24993 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 22:03:25.274923 24993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:25.274935 24993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:25.274948 24993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:25.274959 24993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:25.274971 24993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:25.276744 24993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:25.277063 24993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:25.277338 24993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:25.283893 25001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:25.283972 24993 server_base.cc:1034] running on GCE node
W20260925 22:03:25.284041 24999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.283893 24998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:25.284718 24993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:25.285097 24993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:25.286294 24993 hybrid_clock.cc:648] HybridClock initialized: now 1790373805286267 us; error 53 us; skew 500 ppm
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:25.289871 24993 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:25.290611 24993 webserver.cc:460] Webserver started at http://127.9.112.126:37289/ using document root <none> and password file <none>
I20260925 22:03:25.290884 24993 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:25.290944 24993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:25.291086 24993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:25.292379 24993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "97293ba211d2ebf3247c0613b23e60b9"
server_key_iv: "6cc096567cf095ea6debdf077ef4314b"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.292791 24993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "97293ba211d2ebf3247c0613b23e60b9"
server_key_iv: "6cc096567cf095ea6debdf077ef4314b"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.294785 24993 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 22:03:25.295918 25008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.296283 24993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:03:25.296445 24993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "97293ba211d2ebf3247c0613b23e60b9"
server_key_iv: "6cc096567cf095ea6debdf077ef4314b"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.296545 24993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:25.307727 24993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:25.310024 24993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:25.310170 24993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:25.314567 24993 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40127
I20260925 22:03:25.314664 25060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40127 every 8 connection(s)
I20260925 22:03:25.315030 24993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:25.316107 25061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:25.319029 25061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap starting.
I20260925 22:03:25.319758 25061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:25.320094 25061 log.cc:824] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:25.321216 25061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: No bootstrap required, opened a new log
I20260925 22:03:25.322453 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 24993
I20260925 22:03:25.322556 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:03:25.322880 9665 external_mini_cluster.cc:1428] Setting key bd0311883bf8c1d90e562c3998144a93
I20260925 22:03:25.323240 25061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:25.323479 25061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:25.323521 25061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c289282fb0f45e2bdfb71bfc2ce0e4a, State: Initialized, Role: FOLLOWER
I20260925 22:03:25.323717 25061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:25.323820 25061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:03:25.323855 25061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:03:25.323938 25061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:25.324877 25061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:25.324994 25061 leader_election.cc:302] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c289282fb0f45e2bdfb71bfc2ce0e4a; no voters:
I20260925 22:03:25.325181 25061 leader_election.cc:288] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:03:25.325243 25066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:25.325421 25066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 LEADER]: Becoming Leader. State: Replica: 8c289282fb0f45e2bdfb71bfc2ce0e4a, State: Running, Role: LEADER
I20260925 22:03:25.325523 25061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:25.325646 25066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:25.326149 25067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } } }
I20260925 22:03:25.326261 25067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:25.326524 25068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c289282fb0f45e2bdfb71bfc2ce0e4a. Latest consensus state: current_term: 1 leader_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } } }
I20260925 22:03:25.326884 25068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:25.327145 25075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:25.328203 25075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:25.330147 25075 catalog_manager.cc:1347] Generated new cluster ID: 390a0ae317b8498087d88f66e7fda02b
I20260925 22:03:25.330200 25075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:25.338019 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.323764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.9.112.126:40127)
0925 22:03:25.324010 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.324022 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.324057 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:03:25.324212 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.324221 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.324284 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.324468 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:25.325216 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.326721 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.327808 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.328579 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.328711 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.331862 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:25.331868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:25.331868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:25.331883 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:25.334731 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.335382 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.335385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.335386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.335934 (+ 548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.336247 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.336249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.336249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.336719 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:25.336888 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.337485 (+ 597us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.337741 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":143,"threads_started":1}
I20260925 22:03:25.345651 25075 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:03:25.346459 25075 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:03:25.351305 25075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Generated new TSK 0
I20260925 22:03:25.351572 25075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:03:25.388110 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40127
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:25.491117 25087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:25.491362 25087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:25.491387 25087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:25.492684 25087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:25.492744 25087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:25.492774 25087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:03:25.494334 25087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:25.494577 25087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:25.494786 25087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:25.500404 25092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.500425 25093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.500495 25095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:25.500893 25087 server_base.cc:1034] running on GCE node
I20260925 22:03:25.501139 25087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:25.501372 25087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:25.502532 25087 hybrid_clock.cc:648] HybridClock initialized: now 1790373805502492 us; error 59 us; skew 500 ppm
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:25.505502 25087 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:03:25.506007 25087 webserver.cc:460] Webserver started at http://127.9.112.65:35773/ using document root <none> and password file <none>
I20260925 22:03:25.506229 25087 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:25.506279 25087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:25.506402 25087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:25.507455 25087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "a874ad113fe9433cac3f59d29486a052"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "b8422f9fc910fee4bf719e3a68910b40"
server_key_iv: "0269fe0266a2956ddd078cfdd9d3b7ff"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.507781 25087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "a874ad113fe9433cac3f59d29486a052"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "b8422f9fc910fee4bf719e3a68910b40"
server_key_iv: "0269fe0266a2956ddd078cfdd9d3b7ff"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.509251 25087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:03:25.510075 25102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.510288 25087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:25.510382 25087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "a874ad113fe9433cac3f59d29486a052"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "b8422f9fc910fee4bf719e3a68910b40"
server_key_iv: "0269fe0266a2956ddd078cfdd9d3b7ff"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.510447 25087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:25.525864 25087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:25.528196 25087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:25.528375 25087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:25.528616 25087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:25.528939 25087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:25.528993 25087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.529016 25087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:25.529027 25087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.535658 25087 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:40329
I20260925 22:03:25.535776 25215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:40329 every 8 connection(s)
I20260925 22:03:25.536063 25087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:25.545912 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25087
I20260925 22:03:25.546036 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:03:25.546226 9665 external_mini_cluster.cc:1428] Setting key 926805b5e33ad4ce955bb41042bb216a
I20260925 22:03:25.547067 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.536937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:48921 (local address 127.9.112.126:40127)
0925 22:03:25.537005 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.537007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.537167 (+ 160us) server_negotiation.cc:407] Connection header received
0925 22:03:25.537653 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.537655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.537671 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.537755 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:25.538856 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.539426 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.540667 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.540910 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.540978 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.543613 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:25.543614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:25.543614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:25.543645 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:25.545227 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.545723 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.545725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.545725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.545793 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.546179 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.546180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.546181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.546352 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:25.546434 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.546856 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.546962 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:03:25.547286 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.536872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:48921)
0925 22:03:25.537035 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:25.537079 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:25.537547 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:25.537831 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:25.537835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:25.538450 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:25.538660 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.538667 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.539565 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.539566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.540544 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.540548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.541024 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.541026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.541162 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.541727 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:25.541735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:25.543457 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:25.545316 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:25.545322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:25.545324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:25.545610 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:25.545898 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:25.545898 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:25.545899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:25.546043 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:25.546436 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:25.546453 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:25.546668 (+ 215us) client_negotiation.cc:769] Sending connection context
0925 22:03:25.546794 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 22:03:25.547011 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260925 22:03:25.547951 25216 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40127
I20260925 22:03:25.548067 25216 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:25.548309 25216 heartbeater.cc:511] Master 127.9.112.126:40127 requested a full tablet report, sending...
I20260925 22:03:25.549105 25025 ts_manager.cc:194] Registered new tserver with Master: a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329)
I20260925 22:03:25.549974 25025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:48921
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:03:25.576232 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40127
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:25.657476 25221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:25.657721 25221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:25.657749 25221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:25.658967 25221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:25.659017 25221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:25.659044 25221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:03:25.660816 25221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:25.661082 25221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:25.661324 25221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:25.667074 25226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.667105 25229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.667233 25227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:25.667551 25221 server_base.cc:1034] running on GCE node
I20260925 22:03:25.667824 25221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:25.668141 25221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:25.669313 25221 hybrid_clock.cc:648] HybridClock initialized: now 1790373805669282 us; error 48 us; skew 500 ppm
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:25.672173 25221 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:03:25.672750 25221 webserver.cc:460] Webserver started at http://127.9.112.66:46147/ using document root <none> and password file <none>
I20260925 22:03:25.673013 25221 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:25.673079 25221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:25.673218 25221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:25.674618 25221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "f9ec45463dd5456fbeb079ae18fb382f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "6df7403968ccc257861a2388568d72b4"
server_key_iv: "20eb130a1a74a07548589c7252dc952e"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.675043 25221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "f9ec45463dd5456fbeb079ae18fb382f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "6df7403968ccc257861a2388568d72b4"
server_key_iv: "20eb130a1a74a07548589c7252dc952e"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.676935 25221 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260925 22:03:25.678159 25236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.678424 25221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:25.678540 25221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "f9ec45463dd5456fbeb079ae18fb382f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "6df7403968ccc257861a2388568d72b4"
server_key_iv: "20eb130a1a74a07548589c7252dc952e"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.678638 25221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:25.699587 25221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:25.702282 25221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:25.702446 25221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:25.702732 25221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:25.703089 25221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:25.703125 25221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.703156 25221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:25.703181 25221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.710405 25221 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:42499
I20260925 22:03:25.710497 25349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:42499 every 8 connection(s)
I20260925 22:03:25.710850 25221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:03:25.712692 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25221
I20260925 22:03:25.712786 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:03:25.712956 9665 external_mini_cluster.cc:1428] Setting key 47dd6a1342e6e87dac3009a27ca7589e
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
I20260925 22:03:25.722378 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.711726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:60905 (local address 127.9.112.126:40127)
0925 22:03:25.711852 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.711868 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.712087 (+ 219us) server_negotiation.cc:407] Connection header received
0925 22:03:25.712868 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.712870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.712884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.712938 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:25.714185 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.714797 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.716016 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.716253 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.716334 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.719196 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:25.719198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:25.719198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:25.719206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:25.720862 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.721368 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.721369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.721370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.721427 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.721670 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.721671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.721671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.721800 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:25.721862 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.722160 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.722270 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 22:03:25.722694 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.711635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:60905)
0925 22:03:25.711914 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:25.711972 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:25.712683 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:25.713038 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:25.713044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:25.713848 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:25.714104 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.714111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.714944 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.714947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.715909 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.715914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.716405 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.716408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.716556 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.717149 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:25.717159 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:25.719078 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:25.720951 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:25.720960 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:25.720963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:25.721227 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:25.721491 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:25.721492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:25.721493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:25.721611 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:25.721860 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:25.721868 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:25.722034 (+ 166us) client_negotiation.cc:769] Sending connection context
0925 22:03:25.722131 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 22:03:25.722321 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":128,"threads_started":1}
I20260925 22:03:25.723384 25350 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40127
I20260925 22:03:25.723497 25350 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:25.723692 25350 heartbeater.cc:511] Master 127.9.112.126:40127 requested a full tablet report, sending...
I20260925 22:03:25.724310 25025 ts_manager.cc:194] Registered new tserver with Master: f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499)
I20260925 22:03:25.724754 25025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:60905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:03:25.743650 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40127
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:25.827400 25355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:25.827631 25355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:25.827657 25355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:25.828934 25355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:25.828992 25355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:25.829020 25355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:03:25.830523 25355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:25.830734 25355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:25.830922 25355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:25.836757 25360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.836757 25361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:25.836784 25363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:25.837358 25355 server_base.cc:1034] running on GCE node
I20260925 22:03:25.837615 25355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:25.837929 25355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:25.839118 25355 hybrid_clock.cc:648] HybridClock initialized: now 1790373805839086 us; error 54 us; skew 500 ppm
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:25.842111 25355 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:03:25.842676 25355 webserver.cc:460] Webserver started at http://127.9.112.67:40099/ using document root <none> and password file <none>
I20260925 22:03:25.842933 25355 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:25.842986 25355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:25.843124 25355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:03:25.844434 25355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "d0edbaa1445b4754a44d7b0008d2883f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "5dad044f44ce273d4affb464ea4e69fd"
server_key_iv: "9ab90056a6ad0d200e3fec589448226c"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.844799 25355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "d0edbaa1445b4754a44d7b0008d2883f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "5dad044f44ce273d4affb464ea4e69fd"
server_key_iv: "9ab90056a6ad0d200e3fec589448226c"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.846314 25355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:03:25.847150 25370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.847471 25355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:25.847590 25355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "d0edbaa1445b4754a44d7b0008d2883f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "5dad044f44ce273d4affb464ea4e69fd"
server_key_iv: "9ab90056a6ad0d200e3fec589448226c"
server_key_version: "encryptionkey@0"
I20260925 22:03:25.847666 25355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:25.857136 25355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:25.859727 25355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:25.859875 25355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:25.860136 25355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:25.860677 25355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:25.860729 25355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.860754 25355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:25.860766 25355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:25.867796 25355 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:33285
I20260925 22:03:25.867905 25483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:33285 every 8 connection(s)
I20260925 22:03:25.868216 25355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:03:25.869264 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25355
I20260925 22:03:25.869349 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:03:25.869545 9665 external_mini_cluster.cc:1428] Setting key 77872e656ee40d1760d59e4ec06443d7
Sep 25 22:03:25 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:25.879724 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.869106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:33719 (local address 127.9.112.126:40127)
0925 22:03:25.869208 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.869210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.869902 (+ 692us) server_negotiation.cc:407] Connection header received
0925 22:03:25.870637 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.870638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.870651 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.870717 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:25.871820 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.872389 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.873611 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.873821 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.873886 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.876842 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:25.876844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:25.876844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:25.876853 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:25.878169 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.878641 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.878642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.878643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.878717 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.879012 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.879013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.879013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.879144 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:25.879206 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.879559 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.879642 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 22:03:25.879912 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.869054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:33719)
0925 22:03:25.869741 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:25.869800 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:25.870520 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:25.870790 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:25.870795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:25.871453 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:25.871695 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.871700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.872510 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.872511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.873503 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.873506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.873912 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.873913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.874039 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.874686 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:25.874696 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:25.876677 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:25.878267 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:25.878275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:25.878277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:25.878548 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:25.878800 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:25.878801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:25.878801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:25.878915 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:25.879234 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:25.879241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:25.879408 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 22:03:25.879531 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 22:03:25.879693 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":81,"threads_started":1}
I20260925 22:03:25.880563 25484 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40127
I20260925 22:03:25.880687 25484 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:25.880951 25484 heartbeater.cc:511] Master 127.9.112.126:40127 requested a full tablet report, sending...
I20260925 22:03:25.881501 25025 ts_manager.cc:194] Registered new tserver with Master: d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
I20260925 22:03:25.881781 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:03:25.881942 25025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:33719
I20260925 22:03:25.890484 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.883678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.9.112.126:40127)
0925 22:03:25.883757 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.883760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.883894 (+ 134us) server_negotiation.cc:407] Connection header received
0925 22:03:25.883965 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.883966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.883977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.884024 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:25.884693 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.885232 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.885974 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.886186 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.886244 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.887788 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:25.887790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:25.887790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:25.887798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:25.889310 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.889665 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.889666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.889667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.889732 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:25.889896 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:25.889897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:25.889898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:25.890084 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:25.890179 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.890288 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.890371 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:03:25.892625 25025 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:03:25.894549 25025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:03:25.902127 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.897431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40329 (local address 127.0.0.1:49732)
0925 22:03:25.897646 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:25.897691 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:25.897774 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:25.898116 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:25.898120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:25.898122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:25.898211 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.898213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.899538 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.899539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.900927 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.900930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.901824 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.901825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.901906 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.901919 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 22:03:25.901975 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:03:25.902023 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":162,"thread_start_us":136,"threads_started":1}
I20260925 22:03:25.902148 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.897639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.9.112.65:40329)
0925 22:03:25.897933 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.897938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.897947 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:03:25.897992 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.897994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.898013 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.898093 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:25.898289 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.899409 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.901013 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.901742 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.901821 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.901900 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.902004 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.902054 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260925 22:03:25.902395 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.897434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42499 (local address 127.0.0.1:43662)
0925 22:03:25.897980 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:25.897990 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:25.898046 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:25.898316 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:25.898317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:25.898319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:25.898375 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.898376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.899509 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.899510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.901195 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.901198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.902112 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.902112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.902168 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.902172 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:25.902221 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:03:25.902327 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":77,"thread_start_us":99,"threads_started":1}
I20260925 22:03:25.902638 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.897734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.9.112.66:42499)
0925 22:03:25.897940 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.897944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.898089 (+ 145us) server_negotiation.cc:407] Connection header received
0925 22:03:25.898137 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.898140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.898154 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.898237 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:25.898463 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.899403 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.901287 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.902024 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.902085 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.902135 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.902509 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.902567 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
I20260925 22:03:25.903540 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.897434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:33285 (local address 127.0.0.1:49154)
0925 22:03:25.897884 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:25.897895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:25.897968 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:25.898483 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:25.898484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:25.898486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:25.898533 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.898534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.899991 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.899992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.901211 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:25.901213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.903221 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.903222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:25.903349 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.903354 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:03:25.903418 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 22:03:25.903464 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":69,"threads_started":1}
I20260925 22:03:25.903568 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:25.897649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.9.112.67:33285)
0925 22:03:25.898246 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:25.898251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:25.898261 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:25.898302 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:25.898304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:25.898319 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:25.898406 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:25.898615 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.899801 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.902178 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:25.903135 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:25.903208 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:25.903262 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:25.903439 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 22:03:25.903489 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":131,"threads_started":1}
I20260925 22:03:25.903470 25150 tablet_service.cc:1467] Processing CreateTablet for tablet 23b89377b50b45da9efebc45d2db71bf (DEFAULT_TABLE table=test-table [id=afa22dd3f3964b2baf561889e741c78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:25.903467 25284 tablet_service.cc:1467] Processing CreateTablet for tablet 23b89377b50b45da9efebc45d2db71bf (DEFAULT_TABLE table=test-table [id=afa22dd3f3964b2baf561889e741c78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:25.903890 25150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b89377b50b45da9efebc45d2db71bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:25.903891 25284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b89377b50b45da9efebc45d2db71bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:25.905110 25418 tablet_service.cc:1467] Processing CreateTablet for tablet 23b89377b50b45da9efebc45d2db71bf (DEFAULT_TABLE table=test-table [id=afa22dd3f3964b2baf561889e741c78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:25.905479 25418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b89377b50b45da9efebc45d2db71bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:25.907027 25501 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Bootstrap starting.
I20260925 22:03:25.907749 25502 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Bootstrap starting.
I20260925 22:03:25.907872 25501 tablet_bootstrap.cc:654] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:25.908491 25502 tablet_bootstrap.cc:654] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:25.908550 25501 log.cc:824] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:25.908871 25503 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Bootstrap starting.
I20260925 22:03:25.909042 25502 log.cc:824] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:25.909446 25503 tablet_bootstrap.cc:654] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:25.909716 25501 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: No bootstrap required, opened a new log
I20260925 22:03:25.909763 25501 ts_tablet_manager.cc:1397] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:03:25.909788 25502 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: No bootstrap required, opened a new log
I20260925 22:03:25.909853 25502 ts_tablet_manager.cc:1397] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:25.910041 25503 log.cc:824] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:25.911329 25501 raft_consensus.cc:348] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:25.911477 25501 raft_consensus.cc:374] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:25.911499 25501 raft_consensus.cc:729] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a874ad113fe9433cac3f59d29486a052, State: Initialized, Role: FOLLOWER
I20260925 22:03:25.911468 25502 raft_consensus.cc:348] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:25.911600 25502 raft_consensus.cc:374] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:25.911594 25501 consensus_queue.cc:260] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:25.911633 25502 raft_consensus.cc:729] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ec45463dd5456fbeb079ae18fb382f, State: Initialized, Role: FOLLOWER
I20260925 22:03:25.911739 25502 consensus_queue.cc:260] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:25.911839 25501 ts_tablet_manager.cc:1428] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:25.911876 25216 heartbeater.cc:503] Master 127.9.112.126:40127 was elected leader, sending a full tablet report...
I20260925 22:03:25.912015 25502 ts_tablet_manager.cc:1428] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:25.912031 25350 heartbeater.cc:503] Master 127.9.112.126:40127 was elected leader, sending a full tablet report...
I20260925 22:03:25.912639 25503 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: No bootstrap required, opened a new log
I20260925 22:03:25.912716 25503 ts_tablet_manager.cc:1397] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 22:03:25.914513 25503 raft_consensus.cc:348] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:25.914659 25503 raft_consensus.cc:374] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:25.914680 25503 raft_consensus.cc:729] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0edbaa1445b4754a44d7b0008d2883f, State: Initialized, Role: FOLLOWER
I20260925 22:03:25.914780 25503 consensus_queue.cc:260] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:25.915055 25503 ts_tablet_manager.cc:1428] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:25.915074 25484 heartbeater.cc:503] Master 127.9.112.126:40127 was elected leader, sending a full tablet report...
W20260925 22:03:25.961625 25351 tablet.cc:2369] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:03:26.011543 25509 raft_consensus.cc:482] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:26.011721 25509 raft_consensus.cc:504] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:26.012377 25509 leader_election.cc:288] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329), f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499)
I20260925 22:03:26.016753 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:26.012622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42499 (local address 127.9.112.67:34729)
0925 22:03:26.012686 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:26.012702 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:26.012762 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:26.012962 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:26.012962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:26.012965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:26.013063 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:26.013066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.014051 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.014052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:26.015500 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:26.015503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.016383 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.016385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:26.016483 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:26.016488 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:03:26.016553 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:03:26.016608 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":70}
I20260925 22:03:26.016727 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:26.012758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:34729 (local address 127.9.112.66:42499)
0925 22:03:26.012811 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:26.012813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:26.012823 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:26.012866 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:26.012867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:26.012876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:26.012935 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:26.013128 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.013933 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.015621 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.016239 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.016315 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:26.016327 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:26.016590 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 22:03:26.016651 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:03:26.016808 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:26.012717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:38609 (local address 127.9.112.65:40329)
0925 22:03:26.012811 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:26.012813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:26.012949 (+ 136us) server_negotiation.cc:407] Connection header received
0925 22:03:26.013041 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:26.013042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:26.013049 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:26.013090 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:26.013217 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.014001 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.015612 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.016230 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.016315 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:26.016327 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:26.016689 (+ 362us) server_negotiation.cc:299] Negotiation successful
0925 22:03:26.016742 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 22:03:26.016762 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:26.012623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40329 (local address 127.9.112.67:38609)
0925 22:03:26.012883 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:26.012891 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:26.012942 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:26.013102 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:26.013103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:26.013104 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:26.013166 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:26.013167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.014096 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.014097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:26.015500 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:26.015503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.016514 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.016516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:26.016581 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:26.016584 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:03:26.016630 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:03:26.016667 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":104,"thread_start_us":144,"threads_started":1}
I20260925 22:03:26.017269 25170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b89377b50b45da9efebc45d2db71bf" candidate_uuid: "d0edbaa1445b4754a44d7b0008d2883f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a874ad113fe9433cac3f59d29486a052" is_pre_election: true
I20260925 22:03:26.017406 25304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b89377b50b45da9efebc45d2db71bf" candidate_uuid: "d0edbaa1445b4754a44d7b0008d2883f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ec45463dd5456fbeb079ae18fb382f" is_pre_election: true
I20260925 22:03:26.017519 25170 raft_consensus.cc:2454] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0edbaa1445b4754a44d7b0008d2883f in term 0.
I20260925 22:03:26.017561 25304 raft_consensus.cc:2454] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0edbaa1445b4754a44d7b0008d2883f in term 0.
I20260925 22:03:26.017805 25371 leader_election.cc:302] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a874ad113fe9433cac3f59d29486a052, d0edbaa1445b4754a44d7b0008d2883f; no voters:
I20260925 22:03:26.017957 25509 raft_consensus.cc:2781] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:03:26.018018 25509 raft_consensus.cc:482] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:03:26.018034 25509 raft_consensus.cc:3037] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:26.018899 25509 raft_consensus.cc:504] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:26.019040 25509 leader_election.cc:288] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [CANDIDATE]: Term 1 election: Requested vote from peers a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329), f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499)
I20260925 22:03:26.019210 25304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b89377b50b45da9efebc45d2db71bf" candidate_uuid: "d0edbaa1445b4754a44d7b0008d2883f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ec45463dd5456fbeb079ae18fb382f"
I20260925 22:03:26.019207 25170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b89377b50b45da9efebc45d2db71bf" candidate_uuid: "d0edbaa1445b4754a44d7b0008d2883f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a874ad113fe9433cac3f59d29486a052"
I20260925 22:03:26.019307 25304 raft_consensus.cc:3037] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:26.019338 25170 raft_consensus.cc:3037] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:26.020328 25304 raft_consensus.cc:2454] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0edbaa1445b4754a44d7b0008d2883f in term 1.
I20260925 22:03:26.020328 25170 raft_consensus.cc:2454] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0edbaa1445b4754a44d7b0008d2883f in term 1.
I20260925 22:03:26.020501 25371 leader_election.cc:302] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a874ad113fe9433cac3f59d29486a052, d0edbaa1445b4754a44d7b0008d2883f; no voters:
I20260925 22:03:26.020586 25509 raft_consensus.cc:2781] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:26.020751 25509 raft_consensus.cc:686] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 1 LEADER]: Becoming Leader. State: Replica: d0edbaa1445b4754a44d7b0008d2883f, State: Running, Role: LEADER
I20260925 22:03:26.020859 25509 consensus_queue.cc:237] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:26.021940 25025 catalog_manager.cc:5495] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f reported cstate change: term changed from 0 to 1, leader changed from <none> to d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67). New cstate: current_term: 1 leader_uuid: "d0edbaa1445b4754a44d7b0008d2883f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260925 22:03:26.037031 25217 tablet.cc:2369] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:03:26.053697 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:26.051159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.9.112.67:33285)
0925 22:03:26.051229 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:26.051231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:26.051239 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:03:26.051288 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:26.051288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:26.051309 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:26.051379 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:26.051549 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.052085 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.052896 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:26.053092 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:26.053153 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:26.053312 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:26.053445 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:03:26.053526 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:26.053553 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 22:03:26.053577 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:03:26.055918 25304 raft_consensus.cc:1261] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Refusing update from remote peer d0edbaa1445b4754a44d7b0008d2883f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:26.055963 25170 raft_consensus.cc:1261] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Refusing update from remote peer d0edbaa1445b4754a44d7b0008d2883f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:26.056370 25509 consensus_queue.cc:1035] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:26.056510 25511 consensus_queue.cc:1035] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:26.059846 25516 mvcc.cc:205] Tried to move back new op lower bound from 7333371109601898496 to 7333371109462188032. Current Snapshot: MvccSnapshot[applied={T|T < 7333371109601898496}]
I20260925 22:03:26.059935 25515 mvcc.cc:205] Tried to move back new op lower bound from 7333371109601898496 to 7333371109462188032. Current Snapshot: MvccSnapshot[applied={T|T < 7333371109601898496}]
I20260925 22:03:26.061983 25025 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:26.062083 25025 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:26.062860 25025 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Sending DeleteTablet for 3 replicas of tablet 23b89377b50b45da9efebc45d2db71bf
I20260925 22:03:26.063290 25150 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b89377b50b45da9efebc45d2db71bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:26 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49732
I20260925 22:03:26.063476 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25087
I20260925 22:03:26.063660 25418 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b89377b50b45da9efebc45d2db71bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:26 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:49154
I20260925 22:03:26.064113 25284 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b89377b50b45da9efebc45d2db71bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:26 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:43662
I20260925 22:03:26.064358 25526 tablet_replica.cc:330] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: stopping tablet replica
I20260925 22:03:26.064462 25526 raft_consensus.cc:2229] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:26.064568 25526 raft_consensus.cc:2258] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:26.065093 25526 ts_tablet_manager.cc:1905] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:26.065654 25525 tablet_replica.cc:330] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: stopping tablet replica
I20260925 22:03:26.065796 25525 raft_consensus.cc:2229] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:03:26.065981 25525 raft_consensus.cc:2258] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:26.066522 25525 ts_tablet_manager.cc:1905] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:26.067230 25526 ts_tablet_manager.cc:1918] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:26.067315 25526 log.cc:1196] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/23b89377b50b45da9efebc45d2db71bf
I20260925 22:03:26.067582 25526 ts_tablet_manager.cc:1939] T 23b89377b50b45da9efebc45d2db71bf P f9ec45463dd5456fbeb079ae18fb382f: Deleting consensus metadata
I20260925 22:03:26.068209 25011 catalog_manager.cc:4841] TS f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): tablet 23b89377b50b45da9efebc45d2db71bf (table test-table [id=afa22dd3f3964b2baf561889e741c78a]) successfully deleted
I20260925 22:03:26.068774 25525 ts_tablet_manager.cc:1918] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:26.068843 25525 log.cc:1196] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/23b89377b50b45da9efebc45d2db71bf
I20260925 22:03:26.069077 25525 ts_tablet_manager.cc:1939] T 23b89377b50b45da9efebc45d2db71bf P d0edbaa1445b4754a44d7b0008d2883f: Deleting consensus metadata
I20260925 22:03:26.069769 25012 catalog_manager.cc:4841] TS d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): tablet 23b89377b50b45da9efebc45d2db71bf (table test-table [id=afa22dd3f3964b2baf561889e741c78a]) successfully deleted
W20260925 22:03:26.071168 25009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:40329 (error 108)
I20260925 22:03:26.071394 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25221
I20260925 22:03:26.072363 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:26.071594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40329 (local address 127.0.0.1:49742)
0925 22:03:26.072250 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:26.072304 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622}
W20260925 22:03:26.072494 25009 catalog_manager.cc:4584] TS a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23b89377b50b45da9efebc45d2db71bf: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40329: connect: Connection refused (error 111)
I20260925 22:03:26.078337 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25355
I20260925 22:03:26.083515 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 24993
I20260925 22:03:46.089706 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40127
--webserver_interface=127.9.112.126
--webserver_port=37289
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:03:46.188616 25528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:46.188886 25528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:46.188920 25528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:46.191272 25528 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 22:03:46.191375 25528 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 22:03:46.191390 25528 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 22:03:46.191401 25528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:03:46.191416 25528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:46.191429 25528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:46.191442 25528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:03:46.191454 25528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:03:46.193815 25528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=37289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:46.194159 25528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:46.194465 25528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:46.205566 25535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.205562 25537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:46.205730 25528 server_base.cc:1034] running on GCE node
W20260925 22:03:46.205562 25534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:46.206410 25528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:46.206827 25528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:46.208002 25528 hybrid_clock.cc:648] HybridClock initialized: now 1790373826207988 us; error 39 us; skew 500 ppm
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:46.211575 25528 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:03:46.212177 25528 webserver.cc:460] Webserver started at http://127.9.112.126:37289/ using document root <none> and password file <none>
I20260925 22:03:46.212522 25528 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:46.212596 25528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:46.214326 25528 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:03:46.215196 25544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.215533 25528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.215656 25528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "97293ba211d2ebf3247c0613b23e60b9"
server_key_iv: "6cc096567cf095ea6debdf077ef4314b"
server_key_version: "encryptionkey@0"
I20260925 22:03:46.216224 25528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:46.244053 25528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:46.247043 25528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:46.247205 25528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:46.252746 25528 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40127
I20260925 22:03:46.252849 25596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40127 every 8 connection(s)
I20260925 22:03:46.253314 25528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:03:46.255527 25597 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:03:46.256407 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap starting.
I20260925 22:03:46.256461 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25528
I20260925 22:03:46.256791 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:40329
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:40127
--webserver_port=35773
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:46.265079 25597 log.cc:824] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:46.267077 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:03:46.267403 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap complete.
I20260925 22:03:46.270089 25597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:46.270562 25597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c289282fb0f45e2bdfb71bfc2ce0e4a, State: Initialized, Role: FOLLOWER
I20260925 22:03:46.270745 25597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:46.270828 25597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:03:46.270886 25597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:03:46.270936 25597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 1 FOLLOWER]: Advancing to term 2
I20260925 22:03:46.272091 25597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:46.272200 25597 leader_election.cc:302] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c289282fb0f45e2bdfb71bfc2ce0e4a; no voters:
I20260925 22:03:46.272421 25597 leader_election.cc:288] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 22:03:46.272460 25601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Leader election won for term 2
I20260925 22:03:46.272576 25601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 LEADER]: Becoming Leader. State: Replica: 8c289282fb0f45e2bdfb71bfc2ce0e4a, State: Running, Role: LEADER
I20260925 22:03:46.272688 25601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:03:46.272728 25597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:03:46.273039 25602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c289282fb0f45e2bdfb71bfc2ce0e4a. Latest consensus state: current_term: 2 leader_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } } }
I20260925 22:03:46.273295 25602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:46.273396 25603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } } }
I20260925 22:03:46.273684 25611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:03:46.273711 25603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:03:46.274781 25611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:03:46.274904 25611 catalog_manager.cc:1259] Loaded cluster ID: 390a0ae317b8498087d88f66e7fda02b
I20260925 22:03:46.274941 25611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:03:46.276139 25611 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:03:46.276505 25611 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Loaded TSK: 0
I20260925 22:03:46.288403 25611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Generated new TSK 1
I20260925 22:03:46.288563 25611 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:03:46.355641 25599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:46.355962 25599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:46.356009 25599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:46.357595 25599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:46.357656 25599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:46.357702 25599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:03:46.359530 25599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:40329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=35773
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:46.359856 25599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:46.360119 25599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:46.365980 25624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.366026 25627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.365980 25625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:46.366485 25599 server_base.cc:1034] running on GCE node
I20260925 22:03:46.366796 25599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:46.367244 25599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:46.368466 25599 hybrid_clock.cc:648] HybridClock initialized: now 1790373826368475 us; error 96 us; skew 500 ppm
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:46.371898 25599 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:03:46.372556 25599 webserver.cc:460] Webserver started at http://127.9.112.65:35773/ using document root <none> and password file <none>
I20260925 22:03:46.372910 25599 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:46.372990 25599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:46.374941 25599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.375870 25634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.376058 25599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.376159 25599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "a874ad113fe9433cac3f59d29486a052"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "b8422f9fc910fee4bf719e3a68910b40"
server_key_iv: "0269fe0266a2956ddd078cfdd9d3b7ff"
server_key_version: "encryptionkey@0"
I20260925 22:03:46.376756 25599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:46.403205 25599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:46.405843 25599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:46.406006 25599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:46.406273 25599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:46.406767 25641 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:03:46.408298 25599 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:03:46.408385 25599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 22:03:46.408427 25599 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:03:46.409337 25599 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:03:46.409382 25599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:03:46.409446 25641 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Bootstrap starting.
I20260925 22:03:46.416272 25599 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:40329
I20260925 22:03:46.416673 25748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:40329 every 8 connection(s)
I20260925 22:03:46.416761 25599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:03:46.419516 25641 log.cc:824] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:46.422230 25641 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:03:46.422547 25641 tablet_bootstrap.cc:492] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Bootstrap complete.
I20260925 22:03:46.423605 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25599
I20260925 22:03:46.423624 25641 ts_tablet_manager.cc:1397] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.010s
I20260925 22:03:46.423964 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:42499
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:40127
--webserver_port=46147
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:03:46.425063 25641 raft_consensus.cc:348] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:46.425395 25641 raft_consensus.cc:729] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a874ad113fe9433cac3f59d29486a052, State: Initialized, Role: FOLLOWER
I20260925 22:03:46.425560 25641 consensus_queue.cc:260] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } }
I20260925 22:03:46.425835 25641 ts_tablet_manager.cc:1428] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:46.431841 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.417795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:59177 (local address 127.9.112.126:40127)
0925 22:03:46.418037 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.418048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.420669 (+ 2621us) server_negotiation.cc:407] Connection header received
0925 22:03:46.421373 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.421382 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.421451 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.421616 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:46.422278 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.423655 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.425162 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.425753 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.425836 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.428440 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:46.428446 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:46.428446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:46.428461 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:46.430056 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.430550 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.430552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.430552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.430629 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.430922 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.430923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.430923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.431151 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:46.431216 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.431576 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.431686 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260925 22:03:46.431934 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.417699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:59177)
0925 22:03:46.420466 (+ 2767us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.420517 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.421151 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.421643 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.421646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.421949 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:46.422156 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.422161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.423783 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.423785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.425047 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.425050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.425955 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.425956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.426063 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.426596 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:46.426605 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:46.428253 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:46.430145 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.430153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.430156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.430476 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.430686 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.430687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.430687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.430833 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.431220 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:46.431227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:46.431408 (+ 181us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.431536 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.431724 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2714,"thread_start_us":121,"threads_started":1}
I20260925 22:03:46.432811 25749 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40127
I20260925 22:03:46.432934 25749 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:46.433109 25749 heartbeater.cc:511] Master 127.9.112.126:40127 requested a full tablet report, sending...
I20260925 22:03:46.433923 25561 ts_manager.cc:194] Registered new tserver with Master: a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329)
I20260925 22:03:46.435155 25561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:59177
I20260925 22:03:46.436100 25749 heartbeater.cc:503] Master 127.9.112.126:40127 was elected leader, sending a full tablet report...
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.65@KRBTEST.COM
I20260925 22:03:46.444581 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.434604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40329 (local address 127.0.0.1:41880)
0925 22:03:46.434775 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.434808 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.434878 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.436294 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.436301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.436634 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:46.436688 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.436691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.438002 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.438004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.438660 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.438662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.439290 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.439290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.439357 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.439881 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:46.439886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:46.441191 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:46.443217 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.443226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.443228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.443530 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.443744 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.443745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.443745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.443805 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.444148 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:46.444155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:46.444211 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.444290 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.444499 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260925 22:03:46.444583 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.434684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.9.112.65:40329)
0925 22:03:46.435497 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.435501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.435512 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:03:46.435552 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.435554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.435569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.435641 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:46.436741 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.437898 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.438741 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.439212 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.439279 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.441283 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:46.441293 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:46.441293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:46.441301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:46.443103 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.443607 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.443611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.443611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.443680 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.443861 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.443862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.443862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.444054 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:46.444135 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.444353 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.444476 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":79,"threads_started":1}
I20260925 22:03:46.445240 25683 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b89377b50b45da9efebc45d2db71bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:26 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:41880
I20260925 22:03:46.445802 25758 tablet_replica.cc:330] stopping tablet replica
I20260925 22:03:46.445925 25758 raft_consensus.cc:2229] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:46.446000 25758 raft_consensus.cc:2258] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:46.446355 25758 ts_tablet_manager.cc:1905] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:46.448951 25758 ts_tablet_manager.cc:1918] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:46.449036 25758 log.cc:1196] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal/wals/23b89377b50b45da9efebc45d2db71bf
I20260925 22:03:46.449240 25758 ts_tablet_manager.cc:1939] T 23b89377b50b45da9efebc45d2db71bf P a874ad113fe9433cac3f59d29486a052: Deleting consensus metadata
I20260925 22:03:46.450029 25545 catalog_manager.cc:4841] TS a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329): tablet 23b89377b50b45da9efebc45d2db71bf (table test-table [id=afa22dd3f3964b2baf561889e741c78a]) successfully deleted
W20260925 22:03:46.517144 25754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:46.517373 25754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:46.517400 25754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:46.518644 25754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:46.518702 25754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:46.518731 25754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:03:46.520450 25754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:42499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=46147
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:46.520707 25754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:46.520994 25754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:46.526536 25764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.526652 25763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.526551 25766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:46.526786 25754 server_base.cc:1034] running on GCE node
I20260925 22:03:46.526986 25754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:46.527221 25754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:46.528395 25754 hybrid_clock.cc:648] HybridClock initialized: now 1790373826528374 us; error 31 us; skew 500 ppm
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:46.531325 25754 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:03:46.531909 25754 webserver.cc:460] Webserver started at http://127.9.112.66:46147/ using document root <none> and password file <none>
I20260925 22:03:46.532171 25754 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:46.532232 25754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:46.534210 25754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.535338 25773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.535529 25754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.535625 25754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "f9ec45463dd5456fbeb079ae18fb382f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "6df7403968ccc257861a2388568d72b4"
server_key_iv: "20eb130a1a74a07548589c7252dc952e"
server_key_version: "encryptionkey@0"
I20260925 22:03:46.536099 25754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:46.557250 25754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:46.559373 25754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:46.559496 25754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:46.559688 25754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:46.559933 25754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:46.559963 25754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.559993 25754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:46.560009 25754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.565516 25754 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:42499
I20260925 22:03:46.565554 25886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:42499 every 8 connection(s)
I20260925 22:03:46.565876 25754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:03:46.570106 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25754
I20260925 22:03:46.570381 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:33285
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:40127
--webserver_port=40099
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:46.576232 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.566553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:40491 (local address 127.9.112.126:40127)
0925 22:03:46.566618 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.566621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.566838 (+ 217us) server_negotiation.cc:407] Connection header received
0925 22:03:46.567329 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.567330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.567339 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.567381 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:46.568383 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.568950 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.569964 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.570177 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.570258 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.573131 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:46.573133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:46.573134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:46.573142 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:46.574846 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.575251 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.575252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.575252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.575324 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.575552 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.575553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.575553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.575695 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:46.575760 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.576047 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.576142 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:03:46.576803 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.566541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:40491)
0925 22:03:46.566714 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.566759 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.567231 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.567456 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.567459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.568031 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:46.568226 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.568230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.569037 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.569038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.569878 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.569881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.570315 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.570316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.570475 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.571008 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:46.571018 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:46.573000 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:46.574941 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.574949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.574952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.575176 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.575385 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.575386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.575386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.575486 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.575763 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:46.575766 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:46.575893 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.576401 (+ 508us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.576580 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260925 22:03:46.577312 25887 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40127
I20260925 22:03:46.577412 25887 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:46.577611 25887 heartbeater.cc:511] Master 127.9.112.126:40127 requested a full tablet report, sending...
I20260925 22:03:46.578029 25561 ts_manager.cc:194] Registered new tserver with Master: f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499)
I20260925 22:03:46.578523 25561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:40491
W20260925 22:03:46.653429 25890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:03:46.653656 25890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:03:46.653690 25890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:03:46.654973 25890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:03:46.655023 25890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:03:46.655061 25890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:03:46.656639 25890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:33285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=40099
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:03:46.656894 25890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:03:46.657106 25890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:03:46.662470 25895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.662444 25898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:03:46.662544 25896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:03:46.662523 25890 server_base.cc:1034] running on GCE node
I20260925 22:03:46.662839 25890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:03:46.663098 25890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:03:46.664254 25890 hybrid_clock.cc:648] HybridClock initialized: now 1790373826664241 us; error 28 us; skew 500 ppm
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:03:46.666811 25890 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:03:46.667271 25890 webserver.cc:460] Webserver started at http://127.9.112.67:40099/ using document root <none> and password file <none>
I20260925 22:03:46.667505 25890 fs_manager.cc:359] Metadata directory not provided
I20260925 22:03:46.667567 25890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:03:46.669118 25890 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:03:46.669845 25905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.670096 25890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.670176 25890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "d0edbaa1445b4754a44d7b0008d2883f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "5dad044f44ce273d4affb464ea4e69fd"
server_key_iv: "9ab90056a6ad0d200e3fec589448226c"
server_key_version: "encryptionkey@0"
I20260925 22:03:46.670583 25890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:03:46.686194 25890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:03:46.688612 25890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:03:46.688764 25890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:03:46.689004 25890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:03:46.689301 25890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:03:46.689335 25890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.689363 25890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:03:46.689380 25890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:03:46.694866 25890 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:33285
I20260925 22:03:46.694926 26018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:33285 every 8 connection(s)
I20260925 22:03:46.695242 25890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:03:46.696547 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 25890
Sep 25 22:03:46 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:03:46.703908 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.696928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.9.112.126:40127)
0925 22:03:46.697217 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.697219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.697225 (+ 6us) server_negotiation.cc:407] Connection header received
0925 22:03:46.697242 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.697243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.697253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.697314 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:46.697914 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.698458 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.699227 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.699523 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.699569 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.700995 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:46.700996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:46.700997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:46.701005 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:46.702682 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.703081 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.703083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.703083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.703139 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.703348 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.703348 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.703349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.703484 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:46.703644 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.703743 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.703804 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":269,"thread_start_us":58,"threads_started":1}
I20260925 22:03:46.704934 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.695966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:39599 (local address 127.9.112.126:40127)
0925 22:03:46.696027 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.696029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.696239 (+ 210us) server_negotiation.cc:407] Connection header received
0925 22:03:46.696841 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.696842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.696855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.696905 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:46.697928 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.698449 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.699505 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.699755 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.699832 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.702082 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:46.702083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:46.702083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:46.702090 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:46.703494 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.703974 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.703975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.703975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.704040 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.704286 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.704287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.704288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.704423 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:46.704486 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.704741 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.704819 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":36}
I20260925 22:03:46.705094 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.695963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:39599)
0925 22:03:46.696118 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.696154 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.696742 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.696984 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.696988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.697639 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:03:46.697834 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.697838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.698533 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.698534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.699414 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.699416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.699870 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.699871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.700029 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.700495 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:03:46.700502 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:03:46.702001 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:03:46.703574 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.703581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.703583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.703878 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.704111 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:03:46.704112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:03:46.704112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:03:46.704237 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:03:46.704482 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:03:46.704488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:03:46.704626 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.704717 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.704878 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260925 22:03:46.705533 26019 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40127
I20260925 22:03:46.705623 26019 heartbeater.cc:464] Registering TS with master...
I20260925 22:03:46.705793 26019 heartbeater.cc:511] Master 127.9.112.126:40127 requested a full tablet report, sending...
I20260925 22:03:46.706130 25561 ts_manager.cc:194] Registered new tserver with Master: d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
I20260925 22:03:46.706467 25561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:39599
I20260925 22:03:46.715536 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:03:46.718449 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.715979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.9.112.126:40127)
0925 22:03:46.716046 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.716048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.716116 (+ 68us) server_negotiation.cc:407] Connection header received
0925 22:03:46.716218 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.716219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.716262 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.716327 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:46.716487 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.716979 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.717757 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.717943 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.717996 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.718150 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:46.718198 (+ 48us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 22:03:46.718387 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60588: token expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 22:03:46.718510 25752 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60588: token expired
W20260925 22:03:46.718590 25488 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.112.126:40127)
I20260925 22:03:46.724475 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.718849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.9.112.126:40127)
0925 22:03:46.718903 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.718905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.718912 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:03:46.718930 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.718931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.718939 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.718982 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:03:46.719526 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.720052 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.720841 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.721052 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.721104 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.722203 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:03:46.722204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:03:46.722204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:03:46.722211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:03:46.723435 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.723727 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.723728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.723728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.723789 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:03:46.723960 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:03:46.723961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:03:46.723961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:03:46.724079 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:03:46.724166 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.724267 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.724382 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 22:03:46.725770 25561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:03:46.727519 25561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:03:46.730098 25683 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8c913862054c39b530f0a968614ff8 (DEFAULT_TABLE table=test-table [id=d7a335e0474f4de8978cca65a64c47c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:46.730333 25683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8c913862054c39b530f0a968614ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:46.731953 25641 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Bootstrap starting.
I20260925 22:03:46.732334 25641 tablet_bootstrap.cc:654] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:46.732980 25641 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: No bootstrap required, opened a new log
I20260925 22:03:46.733032 25641 ts_tablet_manager.cc:1397] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.733356 25641 raft_consensus.cc:348] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.733431 25641 raft_consensus.cc:374] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:46.733467 25641 raft_consensus.cc:729] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a874ad113fe9433cac3f59d29486a052, State: Initialized, Role: FOLLOWER
I20260925 22:03:46.733513 25641 consensus_queue.cc:260] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.733585 25641 ts_tablet_manager.cc:1428] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 22:03:46.733656 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.729784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:33285 (local address 127.0.0.1:36684)
0925 22:03:46.729822 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.729836 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.729874 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.730321 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.730322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.730323 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:46.730385 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.730387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.731394 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.731395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.732477 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.732479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.733392 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.733393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.733466 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.733486 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.733535 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.733585 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 22:03:46.733688 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.729983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.9.112.67:33285)
0925 22:03:46.730154 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.730159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.730167 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:03:46.730206 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.730208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.730226 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.730307 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:46.730441 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.731312 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.732575 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.733307 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.733375 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.733437 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.733579 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.733619 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260925 22:03:46.734563 25953 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8c913862054c39b530f0a968614ff8 (DEFAULT_TABLE table=test-table [id=d7a335e0474f4de8978cca65a64c47c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:46.734930 25953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8c913862054c39b530f0a968614ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:46.736002 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.729796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42499 (local address 127.0.0.1:57874)
0925 22:03:46.730169 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.730179 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.730229 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.732474 (+ 2245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.732475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.732477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:46.732521 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.732523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.733753 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.733755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.734798 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.734802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.735765 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.735767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.735847 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.735851 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.735904 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.735942 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260925 22:03:46.736058 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.732000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.9.112.66:42499)
0925 22:03:46.732203 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.732207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.732217 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:03:46.732264 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.732266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.732283 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.732478 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:46.732577 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.733655 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.734966 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.735665 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.735741 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.735801 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.735932 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.735980 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":116,"threads_started":1}
I20260925 22:03:46.736770 25821 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8c913862054c39b530f0a968614ff8 (DEFAULT_TABLE table=test-table [id=d7a335e0474f4de8978cca65a64c47c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:03:46.737056 25821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8c913862054c39b530f0a968614ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:03:46.738219 26029 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Bootstrap starting.
I20260925 22:03:46.738912 26029 tablet_bootstrap.cc:654] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:46.739518 26029 log.cc:824] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:46.740324 26029 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: No bootstrap required, opened a new log
I20260925 22:03:46.740432 26029 ts_tablet_manager.cc:1397] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:03:46.740571 26031 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Bootstrap starting.
I20260925 22:03:46.741138 26031 tablet_bootstrap.cc:654] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Neither blocks nor log segments found. Creating new log.
I20260925 22:03:46.741655 26031 log.cc:824] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:03:46.741945 26029 raft_consensus.cc:348] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.742128 26029 raft_consensus.cc:374] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:46.742170 26029 raft_consensus.cc:729] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0edbaa1445b4754a44d7b0008d2883f, State: Initialized, Role: FOLLOWER
I20260925 22:03:46.742286 26029 consensus_queue.cc:260] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.742466 26031 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: No bootstrap required, opened a new log
I20260925 22:03:46.742532 26031 ts_tablet_manager.cc:1397] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:03:46.742544 26029 ts_tablet_manager.cc:1428] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 22:03:46.742555 26019 heartbeater.cc:503] Master 127.9.112.126:40127 was elected leader, sending a full tablet report...
I20260925 22:03:46.744163 26031 raft_consensus.cc:348] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.744299 26031 raft_consensus.cc:374] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:03:46.744369 26031 raft_consensus.cc:729] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ec45463dd5456fbeb079ae18fb382f, State: Initialized, Role: FOLLOWER
I20260925 22:03:46.744488 26031 consensus_queue.cc:260] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.744748 26031 ts_tablet_manager.cc:1428] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:03:46.744786 25887 heartbeater.cc:503] Master 127.9.112.126:40127 was elected leader, sending a full tablet report...
W20260925 22:03:46.816483 25888 tablet.cc:2369] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:03:46.873160 25755 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:03:46.873302 25755 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.873782 25755 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
I20260925 22:03:46.877056 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.873981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:33285 (local address 127.9.112.65:40807)
0925 22:03:46.874070 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.874081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.874134 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.874282 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.874283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.874285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:46.874401 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.874404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.875111 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.875112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.875956 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.875958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.876754 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.876755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.876845 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.876871 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.876922 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.876967 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":17}
I20260925 22:03:46.877066 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.873985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42499 (local address 127.9.112.65:41695)
0925 22:03:46.874196 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.874205 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:03:46.874255 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:03:46.874368 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:03:46.874368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:03:46.874370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:03:46.874444 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.874446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.875079 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.875080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.875990 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:03:46.875993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.876748 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.876749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:03:46.876847 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.876871 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 22:03:46.876954 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 22:03:46.876982 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":76,"thread_start_us":107,"threads_started":1}
I20260925 22:03:46.877308 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.874030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:41695 (local address 127.9.112.66:42499)
0925 22:03:46.874085 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.874087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.874249 (+ 162us) server_negotiation.cc:407] Connection header received
0925 22:03:46.874304 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.874305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.874314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.874358 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:46.874504 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.874998 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.876074 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.876654 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.876732 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.876741 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.876997 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.877090 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:03:46.877627 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.874076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40807 (local address 127.9.112.67:33285)
0925 22:03:46.874133 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.874135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.874146 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:03:46.874204 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.874204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.874213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.874269 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:03:46.874476 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.875040 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.876051 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.876652 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.876719 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.876728 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.877529 (+ 801us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.877577 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 22:03:46.877650 25841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8c913862054c39b530f0a968614ff8" candidate_uuid: "a874ad113fe9433cac3f59d29486a052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ec45463dd5456fbeb079ae18fb382f" is_pre_election: true
I20260925 22:03:46.877820 25841 raft_consensus.cc:2454] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a874ad113fe9433cac3f59d29486a052 in term 0.
I20260925 22:03:46.877909 25973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8c913862054c39b530f0a968614ff8" candidate_uuid: "a874ad113fe9433cac3f59d29486a052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0edbaa1445b4754a44d7b0008d2883f" is_pre_election: true
I20260925 22:03:46.878049 25973 raft_consensus.cc:2454] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a874ad113fe9433cac3f59d29486a052 in term 0.
I20260925 22:03:46.878118 25637 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a874ad113fe9433cac3f59d29486a052, f9ec45463dd5456fbeb079ae18fb382f; no voters:
I20260925 22:03:46.878238 25755 raft_consensus.cc:2781] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:03:46.878293 25755 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:03:46.878316 25755 raft_consensus.cc:3037] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:46.878996 25755 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.879127 25755 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 1 election: Requested vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
I20260925 22:03:46.879272 25973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8c913862054c39b530f0a968614ff8" candidate_uuid: "a874ad113fe9433cac3f59d29486a052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0edbaa1445b4754a44d7b0008d2883f"
I20260925 22:03:46.879335 25973 raft_consensus.cc:3037] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:46.879992 25973 raft_consensus.cc:2454] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a874ad113fe9433cac3f59d29486a052 in term 1.
I20260925 22:03:46.880101 25841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8c913862054c39b530f0a968614ff8" candidate_uuid: "a874ad113fe9433cac3f59d29486a052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ec45463dd5456fbeb079ae18fb382f"
I20260925 22:03:46.880187 25841 raft_consensus.cc:3037] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:03:46.880223 25638 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a874ad113fe9433cac3f59d29486a052, d0edbaa1445b4754a44d7b0008d2883f; no voters:
I20260925 22:03:46.880402 25755 raft_consensus.cc:2781] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:03:46.880560 25755 raft_consensus.cc:686] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 LEADER]: Becoming Leader. State: Replica: a874ad113fe9433cac3f59d29486a052, State: Running, Role: LEADER
I20260925 22:03:46.880637 25755 consensus_queue.cc:237] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:03:46.881063 25841 raft_consensus.cc:2454] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a874ad113fe9433cac3f59d29486a052 in term 1.
I20260925 22:03:46.881461 25560 catalog_manager.cc:5495] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 reported cstate change: term changed from 0 to 1, leader changed from <none> to a874ad113fe9433cac3f59d29486a052 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "a874ad113fe9433cac3f59d29486a052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } health_report { overall_health: HEALTHY } } }
W20260925 22:03:46.917842 25750 tablet.cc:2369] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:03:46.919956 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.917165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.9.112.65:40329)
0925 22:03:46.917231 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:03:46.917232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:03:46.917241 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:03:46.917277 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:03:46.917278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:03:46.917296 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:03:46.917358 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:03:46.917585 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.918141 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.918954 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:03:46.919218 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:03:46.919292 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:03:46.919483 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:03:46.919621 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:03:46.919686 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:03:46.919843 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 22:03:46.919873 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 22:03:46.921729 25973 raft_consensus.cc:1261] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Refusing update from remote peer a874ad113fe9433cac3f59d29486a052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:46.922061 25841 raft_consensus.cc:1261] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Refusing update from remote peer a874ad113fe9433cac3f59d29486a052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:03:46.922120 25755 consensus_queue.cc:1035] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:46.922322 26036 consensus_queue.cc:1035] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:03:46.925559 26040 mvcc.cc:205] Tried to move back new op lower bound from 7333371195068776448 to 7333371194903887872. Current Snapshot: MvccSnapshot[applied={T|T < 7333371195068776448}]
I20260925 22:03:46.926980 25560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:46.927081 25560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:03:46.927852 25560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Sending DeleteTablet for 3 replicas of tablet 6b8c913862054c39b530f0a968614ff8
I20260925 22:03:46.928126 25683 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b8c913862054c39b530f0a968614ff8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:46 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:41880
I20260925 22:03:46.928167 25821 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b8c913862054c39b530f0a968614ff8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:46 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:57874
I20260925 22:03:46.928220 25758 tablet_replica.cc:330] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: stopping tablet replica
I20260925 22:03:46.928196 25953 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b8c913862054c39b530f0a968614ff8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:03:46 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:36684
I20260925 22:03:46.928270 25758 raft_consensus.cc:2229] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:03:46.928372 25758 raft_consensus.cc:2258] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:46.928443 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25599
I20260925 22:03:46.928982 26049 tablet_replica.cc:330] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: stopping tablet replica
I20260925 22:03:46.929096 26049 raft_consensus.cc:2229] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:46.929198 26049 raft_consensus.cc:2258] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:46.929574 26049 ts_tablet_manager.cc:1905] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:46.931093 26049 ts_tablet_manager.cc:1918] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:46.931160 26049 log.cc:1196] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/6b8c913862054c39b530f0a968614ff8
I20260925 22:03:46.931346 26050 tablet_replica.cc:330] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: stopping tablet replica
I20260925 22:03:46.931378 26049 ts_tablet_manager.cc:1939] T 6b8c913862054c39b530f0a968614ff8 P f9ec45463dd5456fbeb079ae18fb382f: Deleting consensus metadata
I20260925 22:03:46.931465 26050 raft_consensus.cc:2229] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:03:46.931573 26050 raft_consensus.cc:2258] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:03:46.931861 25547 catalog_manager.cc:4841] TS f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): tablet 6b8c913862054c39b530f0a968614ff8 (table test-table [id=d7a335e0474f4de8978cca65a64c47c1]) successfully deleted
I20260925 22:03:46.932032 26050 ts_tablet_manager.cc:1905] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:03:46.933719 26050 ts_tablet_manager.cc:1918] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:03:46.933799 26050 log.cc:1196] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/6b8c913862054c39b530f0a968614ff8
I20260925 22:03:46.934073 26050 ts_tablet_manager.cc:1939] T 6b8c913862054c39b530f0a968614ff8 P d0edbaa1445b4754a44d7b0008d2883f: Deleting consensus metadata
W20260925 22:03:46.934508 25545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:40329 (error 108)
I20260925 22:03:46.934711 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25754
I20260925 22:03:46.934711 25548 catalog_manager.cc:4841] TS d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): tablet 6b8c913862054c39b530f0a968614ff8 (table test-table [id=d7a335e0474f4de8978cca65a64c47c1]) successfully deleted
I20260925 22:03:46.935526 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:03:46.935400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:40329 (local address 127.0.0.1:41890)
0925 22:03:46.935452 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 22:03:46.935497 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 22:03:46.935642 25545 catalog_manager.cc:4584] TS a874ad113fe9433cac3f59d29486a052 (127.9.112.65:40329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b8c913862054c39b530f0a968614ff8: Network error: Client connection negotiation failed: client connection to 127.9.112.65:40329: connect: Connection refused (error 111)
I20260925 22:03:46.939298 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25890
I20260925 22:03:46.943818 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 25528
I20260925 22:04:06.948715 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40127
--webserver_interface=127.9.112.126
--webserver_port=37289
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:07.029907 26052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:07.030150 26052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:07.030172 26052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:07.031379 26052 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 22:04:07.031433 26052 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 22:04:07.031446 26052 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 22:04:07.031456 26052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:07.031468 26052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:07.031481 26052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:07.031491 26052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:07.031503 26052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:07.033054 26052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=37289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:07.033321 26052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:07.033631 26052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:07.039489 26057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.039561 26052 server_base.cc:1034] running on GCE node
W20260925 22:04:07.039500 26058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:07.039500 26060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.040066 26052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:07.040467 26052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:07.041642 26052 hybrid_clock.cc:648] HybridClock initialized: now 1790373847041624 us; error 39 us; skew 500 ppm
Sep 25 22:04:07 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:07.044720 26052 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:07.045269 26052 webserver.cc:460] Webserver started at http://127.9.112.126:37289/ using document root <none> and password file <none>
I20260925 22:04:07.045499 26052 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:07.045553 26052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:07.047246 26052 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:07.048122 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.048396 26052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:07.048501 26052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "97293ba211d2ebf3247c0613b23e60b9"
server_key_iv: "6cc096567cf095ea6debdf077ef4314b"
server_key_version: "encryptionkey@0"
I20260925 22:04:07.049005 26052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:07.062678 26052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:07.065332 26052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:07.065501 26052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:07.071507 26052 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40127
I20260925 22:04:07.071569 26119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40127 every 8 connection(s)
I20260925 22:04:07.071996 26052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:07.073732 26120 sys_catalog.cc:263] Verifying existing consensus state
I20260925 22:04:07.074386 26120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap starting.
I20260925 22:04:07.074944 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26052
I20260925 22:04:07.075240 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:40329
--local_ip_for_outbound_sockets=127.9.112.65
--tserver_master_addrs=127.9.112.126:40127
--webserver_port=35773
--webserver_interface=127.9.112.65
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:04:07.083269 26120 log.cc:824] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:07.087055 26120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:04:07.087535 26120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Bootstrap complete.
I20260925 22:04:07.091689 26120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:04:07.092104 26120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c289282fb0f45e2bdfb71bfc2ce0e4a, State: Initialized, Role: FOLLOWER
I20260925 22:04:07.092309 26120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:04:07.092423 26120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:07.092479 26120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:07.092535 26120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 2 FOLLOWER]: Advancing to term 3
I20260925 22:04:07.094218 26120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:04:07.094364 26120 leader_election.cc:302] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c289282fb0f45e2bdfb71bfc2ce0e4a; no voters:
I20260925 22:04:07.094615 26120 leader_election.cc:288] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 22:04:07.094784 26124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 3 FOLLOWER]: Leader election won for term 3
I20260925 22:04:07.094987 26120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:07.095144 26124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [term 3 LEADER]: Becoming Leader. State: Replica: 8c289282fb0f45e2bdfb71bfc2ce0e4a, State: Running, Role: LEADER
I20260925 22:04:07.095345 26124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } }
I20260925 22:04:07.095765 26126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c289282fb0f45e2bdfb71bfc2ce0e4a. Latest consensus state: current_term: 3 leader_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } } }
I20260925 22:04:07.095743 26125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c289282fb0f45e2bdfb71bfc2ce0e4a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40127 } } }
I20260925 22:04:07.095841 26126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:07.095870 26125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:07.096045 26133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:07.097013 26133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:07.097114 26133 catalog_manager.cc:1259] Loaded cluster ID: 390a0ae317b8498087d88f66e7fda02b
I20260925 22:04:07.097148 26133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:07.098016 26133 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:07.098194 26133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Loaded TSK: 0
I20260925 22:04:07.098234 26133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c289282fb0f45e2bdfb71bfc2ce0e4a: Loaded TSK: 1
F20260925 22:04:07.098444 26133 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7febf174b9ad google::LogMessage::Fail() at ??:0
@ 0x7febf174e88a google::LogMessage::SendToLog() at ??:0
@ 0x7febf174b4de google::LogMessage::Flush() at ??:0
@ 0x7febf174ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7febf3eb8104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7febf3eb5d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7febf96a91cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7febf96a948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7febf96a9594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7febf966fc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7febf9679ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7febf36190d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7febf360e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7febf44726db start_thread at ??:0
@ 0x7febf024961f clone at ??:0
W20260925 22:04:07.178638 26122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:07.178866 26122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:07.178900 26122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:07.180111 26122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:07.180158 26122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:07.180194 26122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:07.181922 26122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:40329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=35773
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:07.182188 26122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:07.182420 26122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:07.188026 26164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:07.188011 26162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.188196 26122 server_base.cc:1034] running on GCE node
W20260925 22:04:07.188043 26161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.188668 26122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:07.189060 26122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:07.190267 26122 hybrid_clock.cc:648] HybridClock initialized: now 1790373847190244 us; error 42 us; skew 500 ppm
Sep 25 22:04:07 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:07.194002 26122 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:04:07.194612 26122 webserver.cc:460] Webserver started at http://127.9.112.65:35773/ using document root <none> and password file <none>
I20260925 22:04:07.194867 26122 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:07.194928 26122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:07.196493 26122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:07.197283 26171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.197556 26122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:07.197649 26122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "a874ad113fe9433cac3f59d29486a052"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "b8422f9fc910fee4bf719e3a68910b40"
server_key_iv: "0269fe0266a2956ddd078cfdd9d3b7ff"
server_key_version: "encryptionkey@0"
I20260925 22:04:07.198010 26122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:07.212222 26122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:07.214473 26122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:07.214618 26122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:07.214857 26122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:07.215391 26178 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 22:04:07.218010 26122 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 22:04:07.218101 26122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260925 22:04:07.218139 26122 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 22:04:07.219039 26122 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 22:04:07.219077 26122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 22:04:07.219231 26178 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Bootstrap starting.
I20260925 22:04:07.226301 26122 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:40329
I20260925 22:04:07.226449 26285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:40329 every 8 connection(s)
I20260925 22:04:07.226744 26122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:07.231889 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26122
I20260925 22:04:07.232190 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:42499
--local_ip_for_outbound_sockets=127.9.112.66
--tserver_master_addrs=127.9.112.126:40127
--webserver_port=46147
--webserver_interface=127.9.112.66
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:04:07.234344 26178 log.cc:824] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:07.238305 26178 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 22:04:07.238830 26178 tablet_bootstrap.cc:492] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Bootstrap complete.
I20260925 22:04:07.240135 26178 ts_tablet_manager.cc:1397] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.012s
I20260925 22:04:07.241967 26178 raft_consensus.cc:348] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:07.242434 26178 raft_consensus.cc:729] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a874ad113fe9433cac3f59d29486a052, State: Initialized, Role: FOLLOWER
I20260925 22:04:07.242617 26178 consensus_queue.cc:260] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:07.242986 26178 ts_tablet_manager.cc:1428] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052: Time spent starting tablet: real 0.003s user 0.004s sys 0.003s
W20260925 22:04:07.314939 26289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:07.315165 26289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:07.315198 26289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:07.316787 26289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:07.316845 26289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:07.316881 26289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:07.318409 26289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:42499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=46147
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:07.318655 26289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:07.318879 26289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:07.320744 26288 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:07.227418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:54547)
0925 22:04:07.227607 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.227652 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:07.228136 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:07.320592 (+ 92456us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
W20260925 22:04:07.321072 26286 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:40127 (0 consecutive failures): Network error: Failed to ping master at 127.9.112.126:40127: Client connection negotiation failed: client connection to 127.9.112.126:40127: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 22:04:07.321537 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.321389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:52793)
0925 22:04:07.321472 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.321513 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 22:04:07.321969 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.321879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:58713)
0925 22:04:07.321928 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.321954 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 22:04:07.322083 26286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:04:07.324909 26297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:07.324909 26296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:07.324923 26299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.325399 26289 server_base.cc:1034] running on GCE node
I20260925 22:04:07.325608 26289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:07.325829 26289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:07.327018 26289 hybrid_clock.cc:648] HybridClock initialized: now 1790373847326998 us; error 36 us; skew 500 ppm
Sep 25 22:04:07 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:07.329772 26289 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:04:07.330255 26289 webserver.cc:460] Webserver started at http://127.9.112.66:46147/ using document root <none> and password file <none>
I20260925 22:04:07.330469 26289 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:07.330520 26289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:07.332022 26289 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:07.332828 26306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.333074 26289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:04:07.333165 26289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "f9ec45463dd5456fbeb079ae18fb382f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "6df7403968ccc257861a2388568d72b4"
server_key_iv: "20eb130a1a74a07548589c7252dc952e"
server_key_version: "encryptionkey@0"
I20260925 22:04:07.333587 26289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:07.345108 26289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:07.347285 26289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:07.347443 26289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:07.347710 26289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:07.348047 26289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:07.348084 26289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.348115 26289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:07.348141 26289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.355168 26289 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:42499
I20260925 22:04:07.355315 26419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:42499 every 8 connection(s)
I20260925 22:04:07.355605 26289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:07.356984 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.356432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:41935)
0925 22:04:07.356649 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.356753 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
W20260925 22:04:07.357474 26420 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:40127 (0 consecutive failures): Network error: Failed to ping master at 127.9.112.126:40127: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
I20260925 22:04:07.357868 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.357722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:49273)
0925 22:04:07.357798 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.357840 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 22:04:07.357973 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26289
I20260925 22:04:07.358449 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:33285
--local_ip_for_outbound_sockets=127.9.112.67
--tserver_master_addrs=127.9.112.126:40127
--webserver_port=40099
--webserver_interface=127.9.112.67
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:04:07.358507 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.358249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:52619)
0925 22:04:07.358448 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.358483 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
W20260925 22:04:07.358695 26420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 22:04:07.449873 26423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:07.450173 26423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:07.450204 26423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:07.451495 26423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:07.451537 26423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:07.451566 26423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:07.453244 26423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:33285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=40099
--tserver_master_addrs=127.9.112.126:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:07.453512 26423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:07.453747 26423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:07.459510 26429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:07.459527 26431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.459520 26423 server_base.cc:1034] running on GCE node
W20260925 22:04:07.459515 26428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:07.460127 26423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:07.460485 26423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:07.461683 26423 hybrid_clock.cc:648] HybridClock initialized: now 1790373847461657 us; error 40 us; skew 500 ppm
Sep 25 22:04:07 dist-test-slave-794d krb5kdc[24980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:07.464763 26423 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:04:07.465319 26423 webserver.cc:460] Webserver started at http://127.9.112.67:40099/ using document root <none> and password file <none>
I20260925 22:04:07.465555 26423 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:07.465608 26423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:07.467262 26423 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:07.468086 26438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.468364 26423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:07.468479 26423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "d0edbaa1445b4754a44d7b0008d2883f"
format_stamp: "Formatted at 2026-09-25 22:03:25 on dist-test-slave-794d"
server_key: "5dad044f44ce273d4affb464ea4e69fd"
server_key_iv: "9ab90056a6ad0d200e3fec589448226c"
server_key_version: "encryptionkey@0"
I20260925 22:04:07.468904 26423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:07.489024 26423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:07.491356 26423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:07.491514 26423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:07.491757 26423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:07.492089 26423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:07.492125 26423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.492156 26423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:07.492172 26423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:07.501543 26423 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:33285
I20260925 22:04:07.501658 26551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:33285 every 8 connection(s)
I20260925 22:04:07.502007 26423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:04:07.503664 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.503119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:35199)
0925 22:04:07.503374 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.503482 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":105,"threads_started":1}
W20260925 22:04:07.504108 26552 heartbeater.cc:644] Failed to heartbeat to 127.9.112.126:40127 (0 consecutive failures): Network error: Failed to ping master at 127.9.112.126:40127: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
I20260925 22:04:07.504627 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.504501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:34665)
0925 22:04:07.504566 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.504603 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 22:04:07.504942 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:07.504871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:48507)
0925 22:04:07.504904 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:07.504928 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260925 22:04:07.505062 26552 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 22:04:07.505249 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26423
I20260925 22:04:08.322798 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.322478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:39987)
0925 22:04:08.322671 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:08.322731 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
I20260925 22:04:08.359486 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.359146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:49219)
0925 22:04:08.359338 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:08.359402 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
I20260925 22:04:08.438469 26558 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:08.438705 26558 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:08.439128 26558 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
Sep 25 22:04:08 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.66@KRBTEST.COM
Sep 25 22:04:08 dist-test-slave-794d krb5kdc[24980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.67@KRBTEST.COM
I20260925 22:04:08.451215 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.439472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:32961 (local address 127.9.112.66:42499)
0925 22:04:08.439717 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:08.439730 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:08.439765 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:04:08.440309 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:08.440311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:08.440408 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:08.440556 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:08.441386 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.442309 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.443536 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.444048 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.444260 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:08.447133 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:08.447140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:08.447141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:08.447158 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:08.449238 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:08.449943 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:08.449946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:08.449946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:08.450041 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:08.450304 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:08.450305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:08.450305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:08.450580 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:08.450657 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:08.450957 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 22:04:08.451083 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260925 22:04:08.451201 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.439329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:42499 (local address 127.9.112.65:32961)
0925 22:04:08.439519 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:08.439533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:08.439609 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:08.440605 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:08.440610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:08.441037 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:08.441263 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:08.441271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.442413 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.442414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:08.443410 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:08.443415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.444234 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.444236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:08.444304 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:08.444972 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:08.444977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:08.446940 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:08.449415 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:08.449425 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:08.449427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:08.449840 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:08.450114 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:08.450115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:08.450115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:08.450241 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:08.450650 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:08.450656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:08.450790 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 22:04:08.450891 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:04:08.451086 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":74}
I20260925 22:04:08.451372 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.439596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:54737 (local address 127.9.112.67:33285)
0925 22:04:08.439770 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:08.439816 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:08.439840 (+ 24us) server_negotiation.cc:407] Connection header received
0925 22:04:08.440297 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:08.440299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:08.440397 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:08.440541 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:08.441408 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.442377 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.443504 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.443955 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.444070 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:08.447499 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:08.447504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:08.447504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:08.447516 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:08.449479 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:08.449905 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:08.449910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:08.449910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:08.450029 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:08.450452 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:08.450455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:08.450455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:08.450820 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:08.450908 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:08.451107 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 22:04:08.451272 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260925 22:04:08.451373 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.439459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:33285 (local address 127.9.112.65:54737)
0925 22:04:08.439649 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:08.439663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:08.439729 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:08.440608 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:08.440610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:08.441102 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:08.441264 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:08.441274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.442488 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.442489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:08.443404 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:08.443409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:08.444055 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:08.444056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:08.444181 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:08.444757 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:08.444770 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:08.447372 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:08.449555 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:08.449556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:08.449557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:08.449816 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:08.450153 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:08.450154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:08.450154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:08.450323 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:08.450901 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:08.450902 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:08.450961 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 22:04:08.451026 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:04:08.451190 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
W20260925 22:04:08.452309 26174 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
W20260925 22:04:08.452451 26175 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
I20260925 22:04:08.452513 26175 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a874ad113fe9433cac3f59d29486a052; no voters: d0edbaa1445b4754a44d7b0008d2883f, f9ec45463dd5456fbeb079ae18fb382f
I20260925 22:04:08.452638 26558 raft_consensus.cc:2726] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:04:08.505757 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:08.505433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:44951)
0925 22:04:08.505640 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:08.505710 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260925 22:04:09.323844 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:09.323525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:45123)
0925 22:04:09.323717 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:09.323782 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260925 22:04:09.360461 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:09.360098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:58311)
0925 22:04:09.360322 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:09.360410 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":139,"threads_started":1}
I20260925 22:04:09.506587 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:09.506298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:46375)
0925 22:04:09.506478 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:09.506538 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260925 22:04:10.106601 26566 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:10.106748 26566 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:10.106935 26566 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
W20260925 22:04:10.107409 26174 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
W20260925 22:04:10.107493 26175 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
I20260925 22:04:10.107585 26175 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a874ad113fe9433cac3f59d29486a052; no voters: d0edbaa1445b4754a44d7b0008d2883f, f9ec45463dd5456fbeb079ae18fb382f
I20260925 22:04:10.107684 26566 raft_consensus.cc:2726] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:04:10.324994 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:10.324610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:43221)
0925 22:04:10.324865 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:10.324946 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":100,"threads_started":1}
I20260925 22:04:10.361420 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:10.361091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:40907)
0925 22:04:10.361291 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:10.361353 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260925 22:04:10.507514 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:10.507184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:38505)
0925 22:04:10.507397 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:10.507461 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
I20260925 22:04:11.326087 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:11.325762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:33005)
0925 22:04:11.325985 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:11.326038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260925 22:04:11.362412 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:11.362131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:44457)
0925 22:04:11.362303 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:11.362348 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260925 22:04:11.508491 26572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:11.508127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:40723)
0925 22:04:11.508318 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:11.508420 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260925 22:04:12.327064 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:12.326719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:51741)
0925 22:04:12.326955 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:12.327016 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260925 22:04:12.363296 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:12.362999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:35067)
0925 22:04:12.363177 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:12.363230 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260925 22:04:12.509467 26575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:12.509143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:59995)
0925 22:04:12.509331 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:12.509400 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260925 22:04:12.972555 26576 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:12.972671 26576 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:12.972930 26576 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
W20260925 22:04:12.973443 26175 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
W20260925 22:04:12.973552 26174 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
I20260925 22:04:12.973594 26174 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a874ad113fe9433cac3f59d29486a052; no voters: d0edbaa1445b4754a44d7b0008d2883f, f9ec45463dd5456fbeb079ae18fb382f
I20260925 22:04:12.973680 26576 raft_consensus.cc:2726] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:04:13.327975 26577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:13.327634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:52065)
0925 22:04:13.327848 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:13.327907 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260925 22:04:13.364110 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:13.363887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:37307)
0925 22:04:13.364028 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:13.364063 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260925 22:04:13.510378 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:13.510098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:38257)
0925 22:04:13.510267 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:13.510319 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260925 22:04:14.328910 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:14.328640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:41935)
0925 22:04:14.328808 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:14.328860 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260925 22:04:14.364954 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:14.364715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:44195)
0925 22:04:14.364862 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:14.364902 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260925 22:04:14.511304 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:14.510956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:35207)
0925 22:04:14.511159 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:14.511217 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
I20260925 22:04:14.851998 26583 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:14.852102 26583 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:14.852268 26583 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
W20260925 22:04:14.852794 26175 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
W20260925 22:04:14.852914 26174 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
I20260925 22:04:14.852972 26174 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a874ad113fe9433cac3f59d29486a052; no voters: d0edbaa1445b4754a44d7b0008d2883f, f9ec45463dd5456fbeb079ae18fb382f
I20260925 22:04:14.853056 26583 raft_consensus.cc:2726] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:04:15.329864 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:15.329532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:54117)
0925 22:04:15.329731 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:15.329800 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260925 22:04:15.365751 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:15.365514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:34205)
0925 22:04:15.365649 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:15.365694 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260925 22:04:15.512567 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:15.512164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:45567)
0925 22:04:15.512413 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:15.512496 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":77,"threads_started":1}
I20260925 22:04:16.330739 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:16.330445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:53037)
0925 22:04:16.330618 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:16.330672 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260925 22:04:16.366578 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:16.366357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:42627)
0925 22:04:16.366497 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:16.366532 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260925 22:04:16.513458 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:16.513186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:36461)
0925 22:04:16.513351 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:16.513410 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260925 22:04:17.331790 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:17.331429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:32987)
0925 22:04:17.331669 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:17.331738 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":86,"threads_started":1}
I20260925 22:04:17.367444 26591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:17.367175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:58903)
0925 22:04:17.367339 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:17.367382 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260925 22:04:17.514377 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:17.514065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:35297)
0925 22:04:17.514243 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:17.514311 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260925 22:04:18.332849 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:18.332491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:34803)
0925 22:04:18.332715 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:18.332792 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260925 22:04:18.368292 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:18.368011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:43643)
0925 22:04:18.368180 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:18.368228 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":73,"threads_started":1}
I20260925 22:04:18.515466 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:18.515135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:50789)
0925 22:04:18.515325 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:18.515395 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260925 22:04:19.333726 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:19.333433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:56933)
0925 22:04:19.333617 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:19.333679 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260925 22:04:19.369206 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:19.368908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:43987)
0925 22:04:19.369077 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:19.369143 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260925 22:04:19.516498 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:19.516146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:41857)
0925 22:04:19.516382 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:19.516446 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":83,"threads_started":1}
I20260925 22:04:19.827864 26599 raft_consensus.cc:482] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:19.827986 26599 raft_consensus.cc:504] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ec45463dd5456fbeb079ae18fb382f" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 42499 } } peers { permanent_uuid: "d0edbaa1445b4754a44d7b0008d2883f" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 33285 } } peers { permanent_uuid: "a874ad113fe9433cac3f59d29486a052" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 40329 } }
I20260925 22:04:19.828246 26599 leader_election.cc:288] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499), d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285)
W20260925 22:04:19.828789 26174 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9ec45463dd5456fbeb079ae18fb382f (127.9.112.66:42499): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
W20260925 22:04:19.829010 26175 leader_election.cc:341] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0edbaa1445b4754a44d7b0008d2883f (127.9.112.67:33285): Not found: Tablet not found: 6b8c913862054c39b530f0a968614ff8
I20260925 22:04:19.829088 26175 leader_election.cc:302] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a874ad113fe9433cac3f59d29486a052; no voters: d0edbaa1445b4754a44d7b0008d2883f, f9ec45463dd5456fbeb079ae18fb382f
I20260925 22:04:19.829154 26599 raft_consensus.cc:2726] T 6b8c913862054c39b530f0a968614ff8 P a874ad113fe9433cac3f59d29486a052 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 22:04:20.334635 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:20.334361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:59695)
0925 22:04:20.334548 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:20.334588 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":117,"threads_started":1}
I20260925 22:04:20.370111 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:20.369801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:58985)
0925 22:04:20.369968 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:20.370043 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260925 22:04:20.517356 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:20.517031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:57431)
0925 22:04:20.517215 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:20.517289 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260925 22:04:21.335659 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:21.335260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:49439)
0925 22:04:21.335496 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:21.335572 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":97,"threads_started":1}
I20260925 22:04:21.371166 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:21.370796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:43791)
0925 22:04:21.371031 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:21.371100 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":95,"threads_started":1}
I20260925 22:04:21.518397 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:21.518022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:37189)
0925 22:04:21.518276 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:21.518341 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260925 22:04:22.336901 26606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:22.336565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.65:56215)
0925 22:04:22.336767 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:22.336853 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260925 22:04:22.372470 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:22.372007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.66:46119)
0925 22:04:22.372242 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:22.372405 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260925 22:04:22.517627 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26122
I20260925 22:04:22.519719 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:22.519347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40127 (local address 127.9.112.67:58509)
0925 22:04:22.519577 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:22.519650 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.126:40127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260925 22:04:22.523350 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26289
I20260925 22:04:22.528445 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26423
2026-09-25T22:04:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58864 ms)
[----------] 30 tests from SecurityITest (234135 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:22 dist-test-slave-794d krb5kdc[26611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:22 dist-test-slave-794d krb5kdc[26611](info): set up 2 sockets
Sep 25 22:04:22 dist-test-slave-794d krb5kdc[26611](info): commencing operation
krb5kdc: starting...
W20260925 22:04:23.876467 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:23 dist-test-slave-794d krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:24.017480 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:43745
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:43745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:24.114723 26624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:24.114997 26624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:24.115033 26624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:24.116334 26624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:24.116421 26624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:24.116439 26624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:24.116457 26624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:24.116482 26624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:24.118160 26624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:43745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:24.118471 26624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:24.118721 26624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:24.124878 26630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.124892 26632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.124938 26629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:24.124904 26624 server_base.cc:1034] running on GCE node
I20260925 22:04:24.125560 26624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:24.125892 26624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:24.127074 26624 hybrid_clock.cc:648] HybridClock initialized: now 1790373864127039 us; error 57 us; skew 500 ppm
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:24.130501 26624 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:24.131127 26624 webserver.cc:460] Webserver started at http://127.9.112.126:43867/ using document root <none> and password file <none>
I20260925 22:04:24.131376 26624 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:24.131428 26624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:24.131562 26624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:24.132874 26624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "10865b34df4f4349b7ee60d7f3dcb36f"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "2522505f5602d8dcf73fe4c665fcb901"
server_key_iv: "dd9e99cbaf69dace345eb89be787f9d1"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.133260 26624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "10865b34df4f4349b7ee60d7f3dcb36f"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "2522505f5602d8dcf73fe4c665fcb901"
server_key_iv: "dd9e99cbaf69dace345eb89be787f9d1"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.134755 26624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:24.135610 26639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.135895 26624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:24.136005 26624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "10865b34df4f4349b7ee60d7f3dcb36f"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "2522505f5602d8dcf73fe4c665fcb901"
server_key_iv: "dd9e99cbaf69dace345eb89be787f9d1"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.136076 26624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:24.153779 26624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:24.156332 26624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:24.156522 26624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:24.161177 26624 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:43745
I20260925 22:04:24.161209 26691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:43745 every 8 connection(s)
I20260925 22:04:24.161615 26624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:24.162604 26692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:24.164580 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26624
I20260925 22:04:24.164677 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:24.164868 9665 external_mini_cluster.cc:1428] Setting key 0f087a757c28f2f6dd15ceec4fd6932b
I20260925 22:04:24.165693 26692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: Bootstrap starting.
I20260925 22:04:24.166370 26692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:24.166731 26692 log.cc:824] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:24.167515 26692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: No bootstrap required, opened a new log
I20260925 22:04:24.169844 26692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43745 } }
I20260925 22:04:24.170010 26692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:24.170046 26692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10865b34df4f4349b7ee60d7f3dcb36f, State: Initialized, Role: FOLLOWER
I20260925 22:04:24.170229 26692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43745 } }
I20260925 22:04:24.170328 26692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:24.170367 26692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:24.170418 26692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:24.171221 26692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43745 } }
I20260925 22:04:24.171332 26692 leader_election.cc:302] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10865b34df4f4349b7ee60d7f3dcb36f; no voters:
I20260925 22:04:24.171504 26692 leader_election.cc:288] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:24.171545 26697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:24.171664 26697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [term 1 LEADER]: Becoming Leader. State: Replica: 10865b34df4f4349b7ee60d7f3dcb36f, State: Running, Role: LEADER
I20260925 22:04:24.171818 26692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:24.171804 26697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43745 } }
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:24.172256 26698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43745 } } }
I20260925 22:04:24.172329 26698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:24.172613 26699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10865b34df4f4349b7ee60d7f3dcb36f. Latest consensus state: current_term: 1 leader_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10865b34df4f4349b7ee60d7f3dcb36f" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43745 } } }
I20260925 22:04:24.172734 26699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f [sys.catalog]: This master's current role is: LEADER
W20260925 22:04:24.173810 26713 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:24.173870 26713 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:24.174057 26703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:24.174252 26703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:24.175796 26703 catalog_manager.cc:1347] Generated new cluster ID: 15aaaadac0ba4fcba1589e6da2727def
I20260925 22:04:24.175843 26703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:24.176788 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.165502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.9.112.126:43745)
0925 22:04:24.165674 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.165683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.165712 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:04:24.166240 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.166248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.166307 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.166457 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:24.167277 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.168309 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.169448 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.169926 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.170048 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.172917 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:24.172922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:24.172923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:24.172937 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:24.175028 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.175469 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.175473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.175473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.175582 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.175829 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.175831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.175831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.176096 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:24.176238 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.176411 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.176562 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260925 22:04:24.194299 26703 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:24.194970 26703 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:24.198447 26703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: Generated new TSK 0
I20260925 22:04:24.198721 26703 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:04:24.236382 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43745
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:24.340931 26718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:24.341208 26718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:24.341236 26718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:24.342520 26718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:24.342568 26718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:24.342607 26718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:24.344143 26718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:24.344420 26718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:24.344663 26718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:24.350327 26724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.350315 26726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:24.350339 26718 server_base.cc:1034] running on GCE node
W20260925 22:04:24.350320 26723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:24.350834 26718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:24.351120 26718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:24.352274 26718 hybrid_clock.cc:648] HybridClock initialized: now 1790373864352254 us; error 40 us; skew 500 ppm
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:24.355196 26718 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:04:24.355724 26718 webserver.cc:460] Webserver started at http://127.9.112.65:37031/ using document root <none> and password file <none>
I20260925 22:04:24.355970 26718 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:24.356020 26718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:24.356153 26718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:24.357393 26718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "163cc0cc3db3453087063dc721eb9946"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "dd11cbe68fe22a8209cf2cc1d4f4f8d4"
server_key_iv: "65dac630cfa57ad0587954652e089339"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.357758 26718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "163cc0cc3db3453087063dc721eb9946"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "dd11cbe68fe22a8209cf2cc1d4f4f8d4"
server_key_iv: "65dac630cfa57ad0587954652e089339"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.359220 26718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:24.360025 26733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.360201 26718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:24.360288 26718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "163cc0cc3db3453087063dc721eb9946"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "dd11cbe68fe22a8209cf2cc1d4f4f8d4"
server_key_iv: "65dac630cfa57ad0587954652e089339"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.360385 26718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:24.377233 26718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:24.379693 26718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:24.379858 26718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:24.380126 26718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:24.380514 26718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:24.380553 26718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.380586 26718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:24.380610 26718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.387670 26718 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:34549
I20260925 22:04:24.387714 26846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:34549 every 8 connection(s)
I20260925 22:04:24.388094 26718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:24.393594 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26718
I20260925 22:04:24.393735 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:04:24.393958 9665 external_mini_cluster.cc:1428] Setting key f73be1cca5c800a823e506ebfeded2fe
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:24.399812 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.389021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:54387 (local address 127.9.112.126:43745)
0925 22:04:24.389125 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.389128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.389186 (+ 58us) server_negotiation.cc:407] Connection header received
0925 22:04:24.389747 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.389748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.389760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.389811 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:24.390923 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.391531 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.392740 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.392984 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.393036 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.396279 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:24.396281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:24.396281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:24.396291 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:24.398102 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.398636 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.398638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.398638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.398722 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.399049 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.399050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.399050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.399205 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:24.399272 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.399582 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.399703 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 22:04:24.399953 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.388889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43745 (local address 127.9.112.65:54387)
0925 22:04:24.389057 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.389101 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.389657 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.389888 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.389891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.390558 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:24.390786 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.390792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.391659 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.391660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.392660 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.392664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.393053 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.393053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.393176 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.393952 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:24.393961 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:24.396171 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:24.398217 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:24.398226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:24.398229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:24.398542 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:24.398817 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:24.398818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:24.398818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:24.398983 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:24.399262 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:24.399267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:24.399453 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.399550 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.399722 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260925 22:04:24.400879 26847 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43745
I20260925 22:04:24.401033 26847 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:24.401295 26847 heartbeater.cc:511] Master 127.9.112.126:43745 requested a full tablet report, sending...
I20260925 22:04:24.401986 26656 ts_manager.cc:194] Registered new tserver with Master: 163cc0cc3db3453087063dc721eb9946 (127.9.112.65:34549)
I20260925 22:04:24.402933 26656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:54387
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:04:24.423858 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43745
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:24.520690 26852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:24.521059 26852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:24.521104 26852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:24.522657 26852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:24.522713 26852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:24.522758 26852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:24.524593 26852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:24.524922 26852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:24.525193 26852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:24.531304 26858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.531323 26860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.531312 26857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:24.531821 26852 server_base.cc:1034] running on GCE node
I20260925 22:04:24.532121 26852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:24.532480 26852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:24.535753 26852 hybrid_clock.cc:648] HybridClock initialized: now 1790373864535721 us; error 43 us; skew 500 ppm
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:24.539657 26852 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:04:24.540406 26852 webserver.cc:460] Webserver started at http://127.9.112.66:40685/ using document root <none> and password file <none>
I20260925 22:04:24.540696 26852 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:24.540809 26852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:24.541002 26852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:24.542289 26852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "6d5ff23777514f1d9d72b908c6e99d1d"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "3d1f46939765419c45ebee3211a507f2"
server_key_iv: "e75c56f0f897b2c68cfbefe808541bdf"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.542658 26852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "6d5ff23777514f1d9d72b908c6e99d1d"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "3d1f46939765419c45ebee3211a507f2"
server_key_iv: "e75c56f0f897b2c68cfbefe808541bdf"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.544198 26852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 22:04:24.545135 26867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.545343 26852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:24.545481 26852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "6d5ff23777514f1d9d72b908c6e99d1d"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "3d1f46939765419c45ebee3211a507f2"
server_key_iv: "e75c56f0f897b2c68cfbefe808541bdf"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.545564 26852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:24.561985 26852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:24.564879 26852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:24.565053 26852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:24.565378 26852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:24.565893 26852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:24.565972 26852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.566026 26852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:24.566056 26852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.575038 26852 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:33941
I20260925 22:04:24.575142 26980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:33941 every 8 connection(s)
I20260925 22:04:24.575538 26852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:24.581959 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26852
I20260925 22:04:24.582098 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:04:24.582342 9665 external_mini_cluster.cc:1428] Setting key 17356cb9bd4f6bb66fc1c4183b8f2dd8
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:24.589340 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.576508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:53491 (local address 127.9.112.126:43745)
0925 22:04:24.576686 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.576689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.576817 (+ 128us) server_negotiation.cc:407] Connection header received
0925 22:04:24.577900 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.577916 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.577932 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.578003 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:24.579260 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.580137 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.581844 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.582149 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.582204 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.585431 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:24.585433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:24.585434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:24.585443 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:24.587412 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.587988 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.587991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.587991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.588073 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.588431 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.588433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.588433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.588614 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:24.588679 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.589089 (+ 410us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.589203 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260925 22:04:24.589612 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.576519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43745 (local address 127.9.112.66:53491)
0925 22:04:24.576691 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.576734 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.577781 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.578076 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.578081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.578835 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:24.579163 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.579171 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.580269 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.580270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.581726 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.581730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.582320 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.582321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.582507 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.583155 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:24.583169 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:24.585328 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:24.587551 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:24.587563 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:24.587567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:24.587900 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:24.588153 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:24.588154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:24.588154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:24.588312 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:24.588688 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:24.588697 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:24.588907 (+ 210us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.589030 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.589273 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260925 22:04:24.590354 26981 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43745
I20260925 22:04:24.590490 26981 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:24.590831 26981 heartbeater.cc:511] Master 127.9.112.126:43745 requested a full tablet report, sending...
I20260925 22:04:24.591336 26656 ts_manager.cc:194] Registered new tserver with Master: 6d5ff23777514f1d9d72b908c6e99d1d (127.9.112.66:33941)
I20260925 22:04:24.591929 26656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:53491
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:04:24.619617 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43745
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:24.716740 26986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:24.717020 26986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:24.717047 26986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:24.718722 26986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:24.718811 26986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:24.718847 26986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:24.720770 26986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:24.721081 26986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:24.721374 26986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:24.727114 26994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.727119 26992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:24.727114 26991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:24.727588 26986 server_base.cc:1034] running on GCE node
I20260925 22:04:24.727895 26986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:24.728240 26986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:24.729416 26986 hybrid_clock.cc:648] HybridClock initialized: now 1790373864729318 us; error 103 us; skew 500 ppm
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:24.732954 26986 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:04:24.733457 26986 webserver.cc:460] Webserver started at http://127.9.112.67:34015/ using document root <none> and password file <none>
I20260925 22:04:24.733729 26986 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:24.733781 26986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:24.733930 26986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:24.735118 26986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "5377d2ff4112475b8a36a6f965f7a917"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "8222ec2b96c350e9dd93db02550b3a6a"
server_key_iv: "71554cd436a1cab4b6ddb3dc570186e5"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.735519 26986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "5377d2ff4112475b8a36a6f965f7a917"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "8222ec2b96c350e9dd93db02550b3a6a"
server_key_iv: "71554cd436a1cab4b6ddb3dc570186e5"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.737072 26986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:24.737929 27001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.738153 26986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:24.738243 26986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "5377d2ff4112475b8a36a6f965f7a917"
format_stamp: "Formatted at 2026-09-25 22:04:24 on dist-test-slave-794d"
server_key: "8222ec2b96c350e9dd93db02550b3a6a"
server_key_iv: "71554cd436a1cab4b6ddb3dc570186e5"
server_key_version: "encryptionkey@0"
I20260925 22:04:24.738312 26986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:24.751324 26986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:24.753890 26986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:24.754099 26986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:24.754367 26986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:24.754710 26986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:24.754746 26986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.754779 26986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:24.754803 26986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:24.761348 26986 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:35403
I20260925 22:04:24.761439 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:35403 every 8 connection(s)
I20260925 22:04:24.761770 26986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:04:24.765920 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 26986
I20260925 22:04:24.766034 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:04:24.766215 9665 external_mini_cluster.cc:1428] Setting key a808c601bce97ac3f7b9f1287f211040
Sep 25 22:04:24 dist-test-slave-794d krb5kdc[26611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:24.772688 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.762637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:47119 (local address 127.9.112.126:43745)
0925 22:04:24.762705 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.762708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.762897 (+ 189us) server_negotiation.cc:407] Connection header received
0925 22:04:24.763592 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.763594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.763609 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.763667 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:24.764847 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.765416 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.766783 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.767053 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.767128 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.769562 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:24.769564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:24.769564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:24.769572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:24.771013 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.771503 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.771505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.771505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.771570 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.771936 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.771937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.771938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.772090 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:24.772155 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.772522 (+ 367us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.772607 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:04:24.773028 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.762577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:43745 (local address 127.9.112.67:47119)
0925 22:04:24.762754 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.762804 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.763471 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.763760 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.763765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.764502 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:24.764748 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.764754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.765510 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.765511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.766555 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.766559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.767136 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.767137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.767305 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.767851 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:24.767860 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:24.769454 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:24.771131 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:24.771139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:24.771142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:24.771413 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:24.771687 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:24.771688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:24.771688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:24.771870 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:24.772174 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:24.772181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:24.772396 (+ 215us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.772506 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.772697 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260925 22:04:24.773535 27115 heartbeater.cc:347] Connected to a master server at 127.9.112.126:43745
I20260925 22:04:24.773644 27115 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:24.773849 27115 heartbeater.cc:511] Master 127.9.112.126:43745 requested a full tablet report, sending...
I20260925 22:04:24.774286 26656 ts_manager.cc:194] Registered new tserver with Master: 5377d2ff4112475b8a36a6f965f7a917 (127.9.112.67:35403)
I20260925 22:04:24.774746 26656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:47119
I20260925 22:04:24.778491 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:04:24.786995 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.780095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.9.112.126:43745)
0925 22:04:24.780159 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.780161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.780251 (+ 90us) server_negotiation.cc:407] Connection header received
0925 22:04:24.780294 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.780294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.780304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.780426 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:24.781015 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.781557 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.782383 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.782604 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.782659 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.784307 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:24.784309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:24.784309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:24.784317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:24.785621 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.786013 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.786015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.786015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.786079 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:24.786285 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:24.786286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:24.786286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:24.786493 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:24.786593 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.786779 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.786905 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:04:24.788808 26656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:04:24.790915 26656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:04:24.798170 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.794108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.9.112.65:34549)
0925 22:04:24.794306 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.794311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.794323 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:04:24.794386 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.794389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.794407 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.794839 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:24.794935 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.795687 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.797094 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.797759 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.797827 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.797898 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.798029 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.798074 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260925 22:04:24.798162 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.793543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34549 (local address 127.0.0.1:44756)
0925 22:04:24.793865 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.793904 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.794055 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.794637 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.794641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.794644 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:24.794756 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.794758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.795778 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.795779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.797004 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.797007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.797835 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.797836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.797921 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.797937 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.798007 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.798071 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":109,"threads_started":1}
I20260925 22:04:24.798271 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.793812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:35403 (local address 127.0.0.1:56842)
0925 22:04:24.794158 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.794166 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.794207 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.794691 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.794692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.794695 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:24.794763 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.794765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.796303 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.796305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.797260 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.797262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.797976 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.797977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.798040 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.798045 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.798092 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.798139 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":108,"threads_started":1}
I20260925 22:04:24.798575 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.794129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.9.112.67:35403)
0925 22:04:24.794409 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.794414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.794426 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:04:24.794559 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.794562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.794580 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.794873 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:24.794937 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.796158 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.797357 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.797894 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.797963 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.798018 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.798206 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.798485 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":89,"threads_started":1}
I20260925 22:04:24.799005 26781 tablet_service.cc:1467] Processing CreateTablet for tablet 905b2bb436e643c2aaaeea26d018e43c (DEFAULT_TABLE table=test-table [id=81e1bb75dc1c4f47b99ee47937240b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:24.799309 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.793543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:33941 (local address 127.0.0.1:60834)
0925 22:04:24.793974 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.793983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.794055 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.794637 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.794641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.794647 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:24.794896 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.794898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.796277 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.796278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.797277 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.797279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.799042 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.799042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.799122 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.799127 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.799183 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.799234 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":135,"thread_start_us":105,"threads_started":1}
I20260925 22:04:24.799445 26781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905b2bb436e643c2aaaeea26d018e43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:24.799732 27049 tablet_service.cc:1467] Processing CreateTablet for tablet 905b2bb436e643c2aaaeea26d018e43c (DEFAULT_TABLE table=test-table [id=81e1bb75dc1c4f47b99ee47937240b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:24.799871 27129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.793837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.9.112.66:33941)
0925 22:04:24.794343 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.794346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.794356 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:04:24.794396 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.794398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.794497 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.794694 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:24.794989 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.796179 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.798059 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.798960 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.799027 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.799083 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.799732 (+ 649us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.799789 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":108,"threads_started":1}
I20260925 22:04:24.800078 27049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905b2bb436e643c2aaaeea26d018e43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:24.800580 26915 tablet_service.cc:1467] Processing CreateTablet for tablet 905b2bb436e643c2aaaeea26d018e43c (DEFAULT_TABLE table=test-table [id=81e1bb75dc1c4f47b99ee47937240b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:24.800927 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905b2bb436e643c2aaaeea26d018e43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:24.802630 27132 tablet_bootstrap.cc:492] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: Bootstrap starting.
I20260925 22:04:24.803278 27132 tablet_bootstrap.cc:654] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:24.803637 27133 tablet_bootstrap.cc:492] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Bootstrap starting.
I20260925 22:04:24.803857 27132 log.cc:824] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:24.804411 27133 tablet_bootstrap.cc:654] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:24.804495 27135 tablet_bootstrap.cc:492] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Bootstrap starting.
I20260925 22:04:24.804725 27132 tablet_bootstrap.cc:492] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: No bootstrap required, opened a new log
I20260925 22:04:24.804777 27132 ts_tablet_manager.cc:1397] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:24.805032 27133 log.cc:824] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:24.805200 27135 tablet_bootstrap.cc:654] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:24.805707 27135 log.cc:824] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:24.805859 27133 tablet_bootstrap.cc:492] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: No bootstrap required, opened a new log
I20260925 22:04:24.805928 27133 ts_tablet_manager.cc:1397] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 22:04:24.806625 27132 raft_consensus.cc:348] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.806842 27132 raft_consensus.cc:374] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:24.806874 27132 raft_consensus.cc:729] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163cc0cc3db3453087063dc721eb9946, State: Initialized, Role: FOLLOWER
I20260925 22:04:24.806854 27135 tablet_bootstrap.cc:492] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: No bootstrap required, opened a new log
I20260925 22:04:24.806931 27135 ts_tablet_manager.cc:1397] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:04:24.806980 27132 consensus_queue.cc:260] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.807258 27132 ts_tablet_manager.cc:1428] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:04:24.807278 26847 heartbeater.cc:503] Master 127.9.112.126:43745 was elected leader, sending a full tablet report...
I20260925 22:04:24.808383 27133 raft_consensus.cc:348] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.808571 27133 raft_consensus.cc:374] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:24.808606 27133 raft_consensus.cc:729] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5377d2ff4112475b8a36a6f965f7a917, State: Initialized, Role: FOLLOWER
I20260925 22:04:24.808728 27133 consensus_queue.cc:260] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.809028 27133 ts_tablet_manager.cc:1428] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260925 22:04:24.809046 27115 heartbeater.cc:503] Master 127.9.112.126:43745 was elected leader, sending a full tablet report...
I20260925 22:04:24.809625 27135 raft_consensus.cc:348] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.809789 27135 raft_consensus.cc:374] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:24.809835 27135 raft_consensus.cc:729] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5ff23777514f1d9d72b908c6e99d1d, State: Initialized, Role: FOLLOWER
I20260925 22:04:24.810125 27135 consensus_queue.cc:260] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.810412 27135 ts_tablet_manager.cc:1428] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:04:24.810454 26981 heartbeater.cc:503] Master 127.9.112.126:43745 was elected leader, sending a full tablet report...
W20260925 22:04:24.826207 26982 tablet.cc:2369] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:04:24.888998 26848 tablet.cc:2369] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:04:24.968906 27138 raft_consensus.cc:482] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:24.969132 27138 raft_consensus.cc:504] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.969535 27138 leader_election.cc:288] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5ff23777514f1d9d72b908c6e99d1d (127.9.112.66:33941), 5377d2ff4112475b8a36a6f965f7a917 (127.9.112.67:35403)
I20260925 22:04:24.973074 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.970161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:51647 (local address 127.9.112.67:35403)
0925 22:04:24.970232 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.970234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.970244 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:04:24.970300 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.970301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.970310 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.970355 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:24.970552 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.971083 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.972238 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.972727 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.972800 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.972813 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.972951 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.972997 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:04:24.973096 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.970057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:35403 (local address 127.9.112.65:51647)
0925 22:04:24.970162 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.970172 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.970236 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.970368 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.970369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.970371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:24.970481 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.970484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.971157 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.971158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.972145 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.972148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.972809 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.972810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.972885 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.972890 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.972934 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.972979 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":73,"thread_start_us":64,"threads_started":1}
I20260925 22:04:24.973151 27129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.969865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:60745 (local address 127.9.112.66:33941)
0925 22:04:24.969958 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.969961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.970091 (+ 130us) server_negotiation.cc:407] Connection header received
0925 22:04:24.970290 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.970292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.970302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.970353 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:24.970562 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.971082 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.972225 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.972801 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.972865 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.972874 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.973022 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.973063 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 22:04:24.973169 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.969776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:33941 (local address 127.9.112.65:60745)
0925 22:04:24.970001 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:24.970014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:24.970103 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:24.970365 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:24.970366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:24.970369 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:24.970495 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.970497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.971162 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.971162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.972146 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:24.972148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.972876 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.972877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:24.972959 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.972963 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:24.973009 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:04:24.973052 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":19,"thread_start_us":109,"threads_started":1}
I20260925 22:04:24.973686 26921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905b2bb436e643c2aaaeea26d018e43c" candidate_uuid: "163cc0cc3db3453087063dc721eb9946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" is_pre_election: true
I20260925 22:04:24.973693 27069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905b2bb436e643c2aaaeea26d018e43c" candidate_uuid: "163cc0cc3db3453087063dc721eb9946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5377d2ff4112475b8a36a6f965f7a917" is_pre_election: true
I20260925 22:04:24.973856 27069 raft_consensus.cc:2454] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163cc0cc3db3453087063dc721eb9946 in term 0.
I20260925 22:04:24.973856 26921 raft_consensus.cc:2454] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163cc0cc3db3453087063dc721eb9946 in term 0.
I20260925 22:04:24.974107 26737 leader_election.cc:302] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163cc0cc3db3453087063dc721eb9946, 5377d2ff4112475b8a36a6f965f7a917; no voters:
I20260925 22:04:24.974267 27138 raft_consensus.cc:2781] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:04:24.974345 27138 raft_consensus.cc:482] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:04:24.974365 27138 raft_consensus.cc:3037] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:24.975291 27138 raft_consensus.cc:504] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.975430 27138 leader_election.cc:288] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [CANDIDATE]: Term 1 election: Requested vote from peers 6d5ff23777514f1d9d72b908c6e99d1d (127.9.112.66:33941), 5377d2ff4112475b8a36a6f965f7a917 (127.9.112.67:35403)
I20260925 22:04:24.975656 26921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905b2bb436e643c2aaaeea26d018e43c" candidate_uuid: "163cc0cc3db3453087063dc721eb9946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5ff23777514f1d9d72b908c6e99d1d"
I20260925 22:04:24.975732 27069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905b2bb436e643c2aaaeea26d018e43c" candidate_uuid: "163cc0cc3db3453087063dc721eb9946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5377d2ff4112475b8a36a6f965f7a917"
I20260925 22:04:24.975762 26921 raft_consensus.cc:3037] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:24.975829 27069 raft_consensus.cc:3037] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:24.976917 27069 raft_consensus.cc:2454] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163cc0cc3db3453087063dc721eb9946 in term 1.
I20260925 22:04:24.976917 26921 raft_consensus.cc:2454] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163cc0cc3db3453087063dc721eb9946 in term 1.
I20260925 22:04:24.977115 26737 leader_election.cc:302] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163cc0cc3db3453087063dc721eb9946, 5377d2ff4112475b8a36a6f965f7a917; no voters:
I20260925 22:04:24.977205 27138 raft_consensus.cc:2781] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:24.977396 27138 raft_consensus.cc:686] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 1 LEADER]: Becoming Leader. State: Replica: 163cc0cc3db3453087063dc721eb9946, State: Running, Role: LEADER
I20260925 22:04:24.977511 27138 consensus_queue.cc:237] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } }
I20260925 22:04:24.978507 26656 catalog_manager.cc:5495] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 163cc0cc3db3453087063dc721eb9946 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "163cc0cc3db3453087063dc721eb9946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "163cc0cc3db3453087063dc721eb9946" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 } health_report { overall_health: UNKNOWN } } }
I20260925 22:04:24.985777 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:24.982758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.9.112.65:34549)
0925 22:04:24.982813 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:24.982816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:24.982825 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:04:24.982905 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:24.982906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:24.982925 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:24.982995 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:24.983220 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.983738 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.984808 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:24.985060 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:24.985130 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:24.985344 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:24.985504 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:24.985586 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:24.985657 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 22:04:24.985679 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:04:24.988144 26921 raft_consensus.cc:1261] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 1 FOLLOWER]: Refusing update from remote peer 163cc0cc3db3453087063dc721eb9946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:24.988144 27069 raft_consensus.cc:1261] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 1 FOLLOWER]: Refusing update from remote peer 163cc0cc3db3453087063dc721eb9946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:24.988576 27138 consensus_queue.cc:1035] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5ff23777514f1d9d72b908c6e99d1d" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 33941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:24.988772 27143 consensus_queue.cc:1035] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5377d2ff4112475b8a36a6f965f7a917" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:24.992192 27148 mvcc.cc:205] Tried to move back new op lower bound from 7333371350988226560 to 7333371350948597760. Current Snapshot: MvccSnapshot[applied={T|T < 7333371350988226560}]
I20260925 22:04:24.993798 26655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:24.993920 26655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:24.994947 26655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10865b34df4f4349b7ee60d7f3dcb36f: Sending DeleteTablet for 3 replicas of tablet 905b2bb436e643c2aaaeea26d018e43c
I20260925 22:04:24.995389 26781 tablet_service.cc:1514] Processing DeleteTablet for tablet 905b2bb436e643c2aaaeea26d018e43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:24 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:44756
I20260925 22:04:24.995533 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet 905b2bb436e643c2aaaeea26d018e43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:24 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:60834
I20260925 22:04:24.995821 27157 tablet_replica.cc:330] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: stopping tablet replica
I20260925 22:04:24.995842 27049 tablet_service.cc:1514] Processing DeleteTablet for tablet 905b2bb436e643c2aaaeea26d018e43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:24 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:56842
I20260925 22:04:24.995965 27157 raft_consensus.cc:2229] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:24.996107 27157 raft_consensus.cc:2258] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:24.996130 27158 tablet_replica.cc:330] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: stopping tablet replica
I20260925 22:04:24.996330 27158 raft_consensus.cc:2229] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:24.996441 27156 tablet_replica.cc:330] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946: stopping tablet replica
I20260925 22:04:24.996555 27156 raft_consensus.cc:2229] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:04:24.996567 27158 raft_consensus.cc:2258] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:24.996615 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26718
I20260925 22:04:24.996671 27156 raft_consensus.cc:2258] T 905b2bb436e643c2aaaeea26d018e43c P 163cc0cc3db3453087063dc721eb9946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:24.997720 27158 ts_tablet_manager.cc:1905] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:24.998808 27157 ts_tablet_manager.cc:1905] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:25.000433 27158 ts_tablet_manager.cc:1918] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:25.000526 27158 log.cc:1196] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/905b2bb436e643c2aaaeea26d018e43c
I20260925 22:04:25.000785 27158 ts_tablet_manager.cc:1939] T 905b2bb436e643c2aaaeea26d018e43c P 5377d2ff4112475b8a36a6f965f7a917: Deleting consensus metadata
I20260925 22:04:25.001004 27157 ts_tablet_manager.cc:1918] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:25.001089 27157 log.cc:1196] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/905b2bb436e643c2aaaeea26d018e43c
I20260925 22:04:25.001273 26643 catalog_manager.cc:4841] TS 5377d2ff4112475b8a36a6f965f7a917 (127.9.112.67:35403): tablet 905b2bb436e643c2aaaeea26d018e43c (table test-table [id=81e1bb75dc1c4f47b99ee47937240b26]) successfully deleted
I20260925 22:04:25.001368 27157 ts_tablet_manager.cc:1939] T 905b2bb436e643c2aaaeea26d018e43c P 6d5ff23777514f1d9d72b908c6e99d1d: Deleting consensus metadata
I20260925 22:04:25.001791 26643 catalog_manager.cc:4841] TS 6d5ff23777514f1d9d72b908c6e99d1d (127.9.112.66:33941): tablet 905b2bb436e643c2aaaeea26d018e43c (table test-table [id=81e1bb75dc1c4f47b99ee47937240b26]) successfully deleted
W20260925 22:04:25.003446 26641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:34549 (error 108)
I20260925 22:04:25.003643 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26852
I20260925 22:04:25.005570 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.005377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34549 (local address 127.0.0.1:44766)
0925 22:04:25.005456 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.005523 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:34549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 22:04:25.005721 26641 catalog_manager.cc:4584] TS 163cc0cc3db3453087063dc721eb9946 (127.9.112.65:34549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 905b2bb436e643c2aaaeea26d018e43c: Network error: Client connection negotiation failed: client connection to 127.9.112.65:34549: connect: Connection refused (error 111)
I20260925 22:04:25.009001 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26986
I20260925 22:04:25.013960 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 26624
2026-09-25T22:04:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2116 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T22:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:25Z Disabled control of system clock
I20260925 22:04:25.042742 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:46479
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:46479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:25.134337 27163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:25.134632 27163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:25.134665 27163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:25.135948 27163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:25.135991 27163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:25.136008 27163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:25.136025 27163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:25.136041 27163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:25.137612 27163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:46479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:25.137910 27163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:25.138248 27163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:25.143914 27169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:25.143916 27171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:25.144224 27163 server_base.cc:1034] running on GCE node
W20260925 22:04:25.143972 27168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:25.144601 27163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:25.144936 27163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:25.146116 27163 hybrid_clock.cc:648] HybridClock initialized: now 1790373865146082 us; error 43 us; skew 500 ppm
I20260925 22:04:25.147709 27163 webserver.cc:460] Webserver started at http://127.9.112.126:41981/ using document root <none> and password file <none>
I20260925 22:04:25.148022 27163 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:25.148082 27163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:25.148228 27163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:25.149542 27163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "ddf0b9be0e70413caacf1f23c4302431"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "129b02b063f3a7b7bad72fbc7298c791"
server_key_iv: "b0fd3e08a9737cb75c7d4dfb9b0c3c78"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.149977 27163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "ddf0b9be0e70413caacf1f23c4302431"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "129b02b063f3a7b7bad72fbc7298c791"
server_key_iv: "b0fd3e08a9737cb75c7d4dfb9b0c3c78"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.151450 27163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:04:25.152277 27177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.152519 27163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:04:25.152614 27163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "ddf0b9be0e70413caacf1f23c4302431"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "129b02b063f3a7b7bad72fbc7298c791"
server_key_iv: "b0fd3e08a9737cb75c7d4dfb9b0c3c78"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.152683 27163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:25.166220 27163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:25.166617 27163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:25.166744 27163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:25.171181 27163 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:46479
I20260925 22:04:25.171223 27229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:46479 every 8 connection(s)
I20260925 22:04:25.171627 27163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:25.172814 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:25.175655 27230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431: Bootstrap starting.
I20260925 22:04:25.176332 27230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:25.176695 27230 log.cc:824] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:25.177490 27230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431: No bootstrap required, opened a new log
I20260925 22:04:25.178061 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27163
I20260925 22:04:25.178151 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:25.178363 9665 external_mini_cluster.cc:1428] Setting key 38b1289a49d98d9d90fd059658b2edbb
I20260925 22:04:25.179733 27230 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf0b9be0e70413caacf1f23c4302431" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 46479 } }
I20260925 22:04:25.179929 27230 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:25.179967 27230 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf0b9be0e70413caacf1f23c4302431, State: Initialized, Role: FOLLOWER
I20260925 22:04:25.180111 27230 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf0b9be0e70413caacf1f23c4302431" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 46479 } }
I20260925 22:04:25.180199 27230 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:25.180239 27230 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:25.180287 27230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:25.181144 27230 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf0b9be0e70413caacf1f23c4302431" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 46479 } }
I20260925 22:04:25.181293 27230 leader_election.cc:302] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddf0b9be0e70413caacf1f23c4302431; no voters:
I20260925 22:04:25.181499 27230 leader_election.cc:288] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:25.181555 27235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:25.181720 27235 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [term 1 LEADER]: Becoming Leader. State: Replica: ddf0b9be0e70413caacf1f23c4302431, State: Running, Role: LEADER
I20260925 22:04:25.181787 27230 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:25.181838 27235 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf0b9be0e70413caacf1f23c4302431" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 46479 } }
I20260925 22:04:25.182289 27236 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddf0b9be0e70413caacf1f23c4302431. Latest consensus state: current_term: 1 leader_uuid: "ddf0b9be0e70413caacf1f23c4302431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf0b9be0e70413caacf1f23c4302431" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 46479 } } }
I20260925 22:04:25.182380 27236 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:25.182598 27242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:25.183418 27242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:25.183465 27237 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddf0b9be0e70413caacf1f23c4302431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf0b9be0e70413caacf1f23c4302431" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 46479 } } }
I20260925 22:04:25.183543 27237 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:25.185254 27242 catalog_manager.cc:1347] Generated new cluster ID: 12c55da948d347b9a5a228b04c6bd3cf
I20260925 22:04:25.185310 27242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:25.185603 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.178994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.9.112.126:46479)
0925 22:04:25.179231 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.179241 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.179284 (+ 43us) server_negotiation.cc:407] Connection header received
0925 22:04:25.179693 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.179703 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.179791 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.179968 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:25.180231 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.182102 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.183212 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.183785 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.183885 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.184737 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:25.184742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:25.184742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:25.184969 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:25.185021 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:25.185036 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:25.185098 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.185304 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.185403 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260925 22:04:25.193615 27242 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:25.194406 27242 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:25.204552 27242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431: Generated new TSK 0
I20260925 22:04:25.204813 27242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:25.209879 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:46479
--builtin_ntp_servers=127.9.112.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:25.300577 27254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:25.300819 27254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:25.300863 27254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:25.302211 27254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:25.302260 27254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:25.302289 27254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:25.303838 27254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:25.304117 27254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:25.304488 27254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:25.310794 27259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:25.310943 27254 server_base.cc:1034] running on GCE node
W20260925 22:04:25.310884 27260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:25.311116 27262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:25.311391 27254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:25.311693 27254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:25.312872 27254 hybrid_clock.cc:648] HybridClock initialized: now 1790373865312843 us; error 49 us; skew 500 ppm
I20260925 22:04:25.314352 27254 webserver.cc:460] Webserver started at http://127.9.112.65:45755/ using document root <none> and password file <none>
I20260925 22:04:25.314613 27254 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:25.314670 27254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:25.314816 27254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:25.316006 27254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "e25790dff7dd4df2bc0839f75a6fe403"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b9f494fe2762bf105f6470051c0b6685"
server_key_iv: "c272138d344349efa5919c7441d49e37"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.316421 27254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "e25790dff7dd4df2bc0839f75a6fe403"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b9f494fe2762bf105f6470051c0b6685"
server_key_iv: "c272138d344349efa5919c7441d49e37"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.317739 27254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:25.318485 27268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.318686 27254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:25.318805 27254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "e25790dff7dd4df2bc0839f75a6fe403"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b9f494fe2762bf105f6470051c0b6685"
server_key_iv: "c272138d344349efa5919c7441d49e37"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.318893 27254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:25.339015 27254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:25.339486 27254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:25.339704 27254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:25.340026 27254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:25.340488 27254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:25.340538 27254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.340564 27254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:25.340575 27254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.346907 27254 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:42721
I20260925 22:04:25.346989 27381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:42721 every 8 connection(s)
I20260925 22:04:25.347322 27254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:25.353392 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.348178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:58457 (local address 127.9.112.126:46479)
0925 22:04:25.348238 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.348241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.348507 (+ 266us) server_negotiation.cc:407] Connection header received
0925 22:04:25.349058 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.349060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.349075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.349140 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:25.349890 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.350430 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.351629 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.351860 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.351926 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.352879 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:25.352880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:25.352880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:25.352886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:25.352906 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:25.352912 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:25.352974 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.353216 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.353283 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:04:25.353415 27384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.348151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:46479 (local address 127.9.112.65:58457)
0925 22:04:25.348380 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.348421 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.348946 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.349201 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.349206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.349217 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:25.349783 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.349788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.350529 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.350530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.351510 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.351514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.351993 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.351994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.352131 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.352715 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:25.352716 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:25.352749 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:25.352777 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:25.353006 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:25.353012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:25.353016 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.353091 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.353208 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":155,"threads_started":1}
I20260925 22:04:25.354399 27382 heartbeater.cc:347] Connected to a master server at 127.9.112.126:46479
I20260925 22:04:25.354517 27382 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:25.354751 27382 heartbeater.cc:511] Master 127.9.112.126:46479 requested a full tablet report, sending...
I20260925 22:04:25.355540 27194 ts_manager.cc:194] Registered new tserver with Master: e25790dff7dd4df2bc0839f75a6fe403 (127.9.112.65:42721)
I20260925 22:04:25.356287 27194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.65:58457
I20260925 22:04:25.356438 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27254
I20260925 22:04:25.356535 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:04:25.356755 9665 external_mini_cluster.cc:1428] Setting key 93debed40d48953a754e5a2f36214caf
I20260925 22:04:25.357683 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:46479
--builtin_ntp_servers=127.9.112.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:25.443259 27385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:25.443482 27385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:25.443506 27385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:25.444792 27385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:25.444839 27385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:25.444865 27385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:25.446290 27385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:25.446534 27385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:25.446856 27385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:25.451858 27391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:25.451905 27393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:25.451946 27390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:25.451996 27385 server_base.cc:1034] running on GCE node
I20260925 22:04:25.452309 27385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:25.452587 27385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:25.453742 27385 hybrid_clock.cc:648] HybridClock initialized: now 1790373865453727 us; error 28 us; skew 500 ppm
I20260925 22:04:25.454946 27385 webserver.cc:460] Webserver started at http://127.9.112.66:34421/ using document root <none> and password file <none>
I20260925 22:04:25.455178 27385 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:25.455230 27385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:25.455358 27385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:25.456570 27385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "8430265578104a598de4984140b79af0"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b146ed198b02d04a60eab5968a466fb2"
server_key_iv: "34ce0709fdaf5054d1573d992b605543"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.456935 27385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "8430265578104a598de4984140b79af0"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b146ed198b02d04a60eab5968a466fb2"
server_key_iv: "34ce0709fdaf5054d1573d992b605543"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.458185 27385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:25.458922 27399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.459086 27385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:04:25.459167 27385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "8430265578104a598de4984140b79af0"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b146ed198b02d04a60eab5968a466fb2"
server_key_iv: "34ce0709fdaf5054d1573d992b605543"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.459228 27385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:25.471532 27385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:25.471881 27385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:25.472038 27385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:25.472298 27385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:25.472688 27385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:25.472726 27385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.472751 27385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:25.472771 27385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.479055 27385 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:39089
I20260925 22:04:25.479101 27512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:39089 every 8 connection(s)
I20260925 22:04:25.479472 27385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:25.482678 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27385
I20260925 22:04:25.482765 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:04:25.482955 9665 external_mini_cluster.cc:1428] Setting key 9b6cc733a128fa604ac09fbca06c4598
I20260925 22:04:25.483970 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:46479
--builtin_ntp_servers=127.9.112.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:04:25.485566 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.480277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:58809 (local address 127.9.112.126:46479)
0925 22:04:25.480380 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.480382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.480655 (+ 273us) server_negotiation.cc:407] Connection header received
0925 22:04:25.481249 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.481250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.481261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.481347 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:25.482090 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.482714 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.483923 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.484219 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.484275 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.485278 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:25.485280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:25.485281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:25.485287 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:25.485305 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:25.485310 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:25.485377 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.485441 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.485476 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:04:25.485726 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.480251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:46479 (local address 127.9.112.66:58809)
0925 22:04:25.480500 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.480543 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.481052 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.481428 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.481432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.481441 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:25.481977 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.481982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.482838 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.482839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.483848 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.483851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.484306 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.484307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.484508 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.485077 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:25.485078 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:25.485128 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:25.485164 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:25.485383 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:25.485386 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:25.485389 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.485441 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.485532 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":134,"threads_started":1}
I20260925 22:04:25.486222 27513 heartbeater.cc:347] Connected to a master server at 127.9.112.126:46479
I20260925 22:04:25.486318 27513 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:25.486505 27513 heartbeater.cc:511] Master 127.9.112.126:46479 requested a full tablet report, sending...
I20260925 22:04:25.486862 27194 ts_manager.cc:194] Registered new tserver with Master: 8430265578104a598de4984140b79af0 (127.9.112.66:39089)
I20260925 22:04:25.487221 27194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.66:58809
W20260925 22:04:25.565702 27516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:25.565917 27516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:25.565940 27516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:25.567126 27516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:25.567170 27516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:25.567203 27516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:25.568621 27516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:25.568859 27516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:25.569144 27516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:04:25.574390 27516 server_base.cc:1034] running on GCE node
W20260925 22:04:25.574352 27521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:25.574353 27524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:25.574370 27522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:25.574820 27516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:25.575066 27516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:25.576332 27516 hybrid_clock.cc:648] HybridClock initialized: now 1790373865576317 us; error 31 us; skew 500 ppm
I20260925 22:04:25.577674 27516 webserver.cc:460] Webserver started at http://127.9.112.67:40973/ using document root <none> and password file <none>
I20260925 22:04:25.577926 27516 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:25.577973 27516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:25.578085 27516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:25.579128 27516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "937da6302d6c49b4beba1758dc01c6c9"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b35bf20000b3e3941d91ce1aed505961"
server_key_iv: "ced652a84e85402b727edd5e3705254c"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.579461 27516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "937da6302d6c49b4beba1758dc01c6c9"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b35bf20000b3e3941d91ce1aed505961"
server_key_iv: "ced652a84e85402b727edd5e3705254c"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.580735 27516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:04:25.581519 27530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.581660 27516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.581740 27516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "937da6302d6c49b4beba1758dc01c6c9"
format_stamp: "Formatted at 2026-09-25 22:04:25 on dist-test-slave-794d"
server_key: "b35bf20000b3e3941d91ce1aed505961"
server_key_iv: "ced652a84e85402b727edd5e3705254c"
server_key_version: "encryptionkey@0"
I20260925 22:04:25.581810 27516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:25.593984 27516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:25.594300 27516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:25.594429 27516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:25.594637 27516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:25.594949 27516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:25.594981 27516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.595011 27516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:25.595029 27516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:25.600659 27516 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:44353
I20260925 22:04:25.600713 27643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:44353 every 8 connection(s)
I20260925 22:04:25.601049 27516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:04:25.606114 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.601747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:48895 (local address 127.9.112.126:46479)
0925 22:04:25.601793 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.601794 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.602024 (+ 230us) server_negotiation.cc:407] Connection header received
0925 22:04:25.602509 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.602510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.602522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.602577 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:25.603203 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.603705 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.604794 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.605002 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.605063 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.605845 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:25.605846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:25.605847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:25.605851 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:25.605866 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:25.605870 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:25.605927 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.606018 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.606050 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:04:25.606235 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.601743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:46479 (local address 127.9.112.67:48895)
0925 22:04:25.601913 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.601946 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.602417 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.602639 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.602642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.602650 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:25.603138 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.603142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.603799 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.603799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.604683 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.604686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.605070 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.605071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.605177 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.605699 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:25.605700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:25.605729 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:25.605754 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:25.605930 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:25.605932 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:25.605934 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.605980 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.606060 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260925 22:04:25.606846 27644 heartbeater.cc:347] Connected to a master server at 127.9.112.126:46479
I20260925 22:04:25.606931 27644 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:25.607095 27644 heartbeater.cc:511] Master 127.9.112.126:46479 requested a full tablet report, sending...
I20260925 22:04:25.607431 27194 ts_manager.cc:194] Registered new tserver with Master: 937da6302d6c49b4beba1758dc01c6c9 (127.9.112.67:44353)
I20260925 22:04:25.607802 27194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.67:48895
I20260925 22:04:25.609309 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27516
I20260925 22:04:25.609414 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:04:25.609583 9665 external_mini_cluster.cc:1428] Setting key 9971d82a2a99c9be37bbe430c77a734b
I20260925 22:04:25.610462 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:04:25.614832 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.611955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.9.112.126:46479)
0925 22:04:25.612009 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.612011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.612120 (+ 109us) server_negotiation.cc:407] Connection header received
0925 22:04:25.612193 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.612194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.612207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.612253 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:25.612431 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.612927 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.613723 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.613933 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.613987 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.614575 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:25.614576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:25.614576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:25.614581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:25.614595 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:25.614599 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:25.614670 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.614727 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.614752 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:04:25.616466 27194 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:04:25.618155 27194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:04:25.624722 27655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.620462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.9.112.66:39089)
0925 22:04:25.620713 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.620717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.620770 (+ 53us) server_negotiation.cc:407] Connection header received
0925 22:04:25.620930 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.620932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.620949 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.621016 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:25.621246 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.622468 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.623520 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.624300 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.624387 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.624481 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.624593 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.624637 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
I20260925 22:04:25.624724 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.620310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:39089 (local address 127.0.0.1:57392)
0925 22:04:25.620671 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.620716 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.620793 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.621092 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.621095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.621107 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:25.621193 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.621195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.622544 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.622545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.623420 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.623423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.624413 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.624415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.624491 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.624507 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.624572 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.624626 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":37,"thread_start_us":94,"threads_started":1}
I20260925 22:04:25.624888 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.620436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.9.112.65:42721)
0925 22:04:25.620634 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.620639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.620786 (+ 147us) server_negotiation.cc:407] Connection header received
0925 22:04:25.620935 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.620938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.620955 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.621009 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:25.621303 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.622415 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.623739 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.624449 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.624505 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.624579 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.624774 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.624813 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260925 22:04:25.625088 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.620314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42721 (local address 127.0.0.1:52324)
0925 22:04:25.620682 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.620725 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.620817 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.621097 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.621098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.621100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:25.621211 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.621212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.622497 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.622498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.623678 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.623680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.624621 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.624622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.624682 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.624686 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.624985 (+ 299us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.625032 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":35,"thread_start_us":148,"threads_started":1}
I20260925 22:04:25.625476 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.620750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41352 (local address 127.9.112.67:44353)
0925 22:04:25.620925 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.620928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.620978 (+ 50us) server_negotiation.cc:407] Connection header received
0925 22:04:25.621046 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.621047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.621061 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.621133 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:25.621286 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.622177 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.623898 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.624437 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.624494 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.624577 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.624775 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.625416 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260925 22:04:25.625651 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.620554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:44353 (local address 127.0.0.1:41352)
0925 22:04:25.620924 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.620933 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.620985 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.621128 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.621129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.621131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:25.621205 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.621207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.622247 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.622248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.623833 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.623835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.624608 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.624609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.624656 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.624659 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.624703 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.624737 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":100,"thread_start_us":68,"threads_started":1}
I20260925 22:04:25.625838 27447 tablet_service.cc:1467] Processing CreateTablet for tablet 770b1373ef7d466f91aab7053518b5fe (DEFAULT_TABLE table=test-table [id=57c21e84832d45af9c88747711309548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:25.626237 27447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770b1373ef7d466f91aab7053518b5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:25.626359 27578 tablet_service.cc:1467] Processing CreateTablet for tablet 770b1373ef7d466f91aab7053518b5fe (DEFAULT_TABLE table=test-table [id=57c21e84832d45af9c88747711309548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:25.626664 27578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770b1373ef7d466f91aab7053518b5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:25.628861 27316 tablet_service.cc:1467] Processing CreateTablet for tablet 770b1373ef7d466f91aab7053518b5fe (DEFAULT_TABLE table=test-table [id=57c21e84832d45af9c88747711309548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:25.629222 27316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770b1373ef7d466f91aab7053518b5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:25.629762 27658 tablet_bootstrap.cc:492] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Bootstrap starting.
I20260925 22:04:25.629976 27659 tablet_bootstrap.cc:492] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Bootstrap starting.
I20260925 22:04:25.630669 27659 tablet_bootstrap.cc:654] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:25.631006 27659 log.cc:824] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:25.631729 27659 tablet_bootstrap.cc:492] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: No bootstrap required, opened a new log
I20260925 22:04:25.631794 27659 ts_tablet_manager.cc:1397] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:04:25.631853 27658 tablet_bootstrap.cc:654] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:25.632200 27658 log.cc:824] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:25.632571 27662 tablet_bootstrap.cc:492] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403: Bootstrap starting.
I20260925 22:04:25.633087 27662 tablet_bootstrap.cc:654] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:25.633349 27662 log.cc:824] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:25.633992 27662 tablet_bootstrap.cc:492] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403: No bootstrap required, opened a new log
I20260925 22:04:25.633919 27659 raft_consensus.cc:348] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.634047 27659 raft_consensus.cc:374] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:25.634051 27662 ts_tablet_manager.cc:1397] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:04:25.634066 27659 raft_consensus.cc:729] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937da6302d6c49b4beba1758dc01c6c9, State: Initialized, Role: FOLLOWER
I20260925 22:04:25.634189 27659 consensus_queue.cc:260] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.634442 27659 ts_tablet_manager.cc:1428] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:04:25.634498 27644 heartbeater.cc:503] Master 127.9.112.126:46479 was elected leader, sending a full tablet report...
I20260925 22:04:25.635041 27658 tablet_bootstrap.cc:492] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: No bootstrap required, opened a new log
I20260925 22:04:25.635104 27658 ts_tablet_manager.cc:1397] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 22:04:25.637655 27658 raft_consensus.cc:348] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.637854 27658 raft_consensus.cc:374] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:25.637890 27658 raft_consensus.cc:729] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430265578104a598de4984140b79af0, State: Initialized, Role: FOLLOWER
I20260925 22:04:25.638007 27658 consensus_queue.cc:260] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.638311 27658 ts_tablet_manager.cc:1428] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 22:04:25.638429 27513 heartbeater.cc:503] Master 127.9.112.126:46479 was elected leader, sending a full tablet report...
I20260925 22:04:25.638437 27662 raft_consensus.cc:348] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.638566 27662 raft_consensus.cc:374] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:25.638583 27662 raft_consensus.cc:729] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25790dff7dd4df2bc0839f75a6fe403, State: Initialized, Role: FOLLOWER
I20260925 22:04:25.638713 27662 consensus_queue.cc:260] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.638972 27662 ts_tablet_manager.cc:1428] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 22:04:25.639055 27382 heartbeater.cc:503] Master 127.9.112.126:46479 was elected leader, sending a full tablet report...
I20260925 22:04:25.675623 27664 raft_consensus.cc:482] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:25.675755 27664 raft_consensus.cc:504] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.676397 27664 leader_election.cc:288] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430265578104a598de4984140b79af0 (127.9.112.66:39089), e25790dff7dd4df2bc0839f75a6fe403 (127.9.112.65:42721)
I20260925 22:04:25.679322 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.676332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42721 (local address 127.9.112.67:39515)
0925 22:04:25.676505 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.676517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.676565 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.676730 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.676731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.676732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:25.676808 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.676810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.677577 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.677579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.678328 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.678330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.678999 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.679001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.679076 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.679080 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.679196 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.679243 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260925 22:04:25.679327 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.676483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:39515 (local address 127.9.112.65:42721)
0925 22:04:25.676573 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.676575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.676639 (+ 64us) server_negotiation.cc:407] Connection header received
0925 22:04:25.676667 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.676667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.676679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.676725 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:25.676896 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.677507 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.678413 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.678911 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.678982 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.678990 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.679215 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.679258 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 22:04:25.679517 27655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.676650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:52893 (local address 127.9.112.66:39089)
0925 22:04:25.676691 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.676693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.676813 (+ 120us) server_negotiation.cc:407] Connection header received
0925 22:04:25.677008 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.677010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.677020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.677106 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:25.677280 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.677724 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.678605 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.679154 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.679215 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.679222 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.679412 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.679460 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:04:25.679747 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.676332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:39089 (local address 127.9.112.67:52893)
0925 22:04:25.676695 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.676728 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:25.676845 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:25.677110 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:25.677111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:25.677113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:25.677192 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.677195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.677794 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.677794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.678532 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:25.678534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.679262 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.679263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:25.679331 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.679335 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:25.679615 (+ 280us) client_negotiation.cc:240] Negotiation successful
0925 22:04:25.679660 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":35,"thread_start_us":91,"threads_started":1}
I20260925 22:04:25.679724 27336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770b1373ef7d466f91aab7053518b5fe" candidate_uuid: "937da6302d6c49b4beba1758dc01c6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e25790dff7dd4df2bc0839f75a6fe403" is_pre_election: true
I20260925 22:04:25.679879 27336 raft_consensus.cc:2454] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937da6302d6c49b4beba1758dc01c6c9 in term 0.
I20260925 22:04:25.680089 27533 leader_election.cc:302] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 937da6302d6c49b4beba1758dc01c6c9, e25790dff7dd4df2bc0839f75a6fe403; no voters:
I20260925 22:04:25.680089 27467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770b1373ef7d466f91aab7053518b5fe" candidate_uuid: "937da6302d6c49b4beba1758dc01c6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430265578104a598de4984140b79af0" is_pre_election: true
I20260925 22:04:25.680208 27467 raft_consensus.cc:2454] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937da6302d6c49b4beba1758dc01c6c9 in term 0.
I20260925 22:04:25.680217 27664 raft_consensus.cc:2781] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:04:25.680265 27664 raft_consensus.cc:482] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:04:25.680294 27664 raft_consensus.cc:3037] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:25.681056 27664 raft_consensus.cc:504] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.681191 27664 leader_election.cc:288] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [CANDIDATE]: Term 1 election: Requested vote from peers 8430265578104a598de4984140b79af0 (127.9.112.66:39089), e25790dff7dd4df2bc0839f75a6fe403 (127.9.112.65:42721)
I20260925 22:04:25.681435 27336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770b1373ef7d466f91aab7053518b5fe" candidate_uuid: "937da6302d6c49b4beba1758dc01c6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e25790dff7dd4df2bc0839f75a6fe403"
I20260925 22:04:25.681525 27336 raft_consensus.cc:3037] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:25.682317 27336 raft_consensus.cc:2454] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937da6302d6c49b4beba1758dc01c6c9 in term 1.
I20260925 22:04:25.682521 27533 leader_election.cc:302] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 937da6302d6c49b4beba1758dc01c6c9, e25790dff7dd4df2bc0839f75a6fe403; no voters:
I20260925 22:04:25.682559 27467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770b1373ef7d466f91aab7053518b5fe" candidate_uuid: "937da6302d6c49b4beba1758dc01c6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430265578104a598de4984140b79af0"
I20260925 22:04:25.682677 27664 raft_consensus.cc:2781] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:25.682662 27467 raft_consensus.cc:3037] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:25.682838 27664 raft_consensus.cc:686] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 1 LEADER]: Becoming Leader. State: Replica: 937da6302d6c49b4beba1758dc01c6c9, State: Running, Role: LEADER
I20260925 22:04:25.682955 27664 consensus_queue.cc:237] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } }
I20260925 22:04:25.683601 27467 raft_consensus.cc:2454] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937da6302d6c49b4beba1758dc01c6c9 in term 1.
I20260925 22:04:25.683985 27194 catalog_manager.cc:5495] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 937da6302d6c49b4beba1758dc01c6c9 (127.9.112.67). New cstate: current_term: 1 leader_uuid: "937da6302d6c49b4beba1758dc01c6c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "937da6302d6c49b4beba1758dc01c6c9" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 44353 } health_report { overall_health: HEALTHY } } }
I20260925 22:04:25.699254 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.696531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.9.112.67:44353)
0925 22:04:25.696573 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:25.696575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:25.696586 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:25.696624 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:25.696625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:25.696643 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:25.696717 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:25.696862 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.697399 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.698203 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:25.698417 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:25.698483 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:25.698748 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:25.698886 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:25.698965 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:25.699138 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 22:04:25.699160 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 22:04:25.701198 27336 raft_consensus.cc:1261] T 770b1373ef7d466f91aab7053518b5fe P e25790dff7dd4df2bc0839f75a6fe403 [term 1 FOLLOWER]: Refusing update from remote peer 937da6302d6c49b4beba1758dc01c6c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:25.701198 27467 raft_consensus.cc:1261] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 1 FOLLOWER]: Refusing update from remote peer 937da6302d6c49b4beba1758dc01c6c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:25.701577 27668 consensus_queue.cc:1035] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e25790dff7dd4df2bc0839f75a6fe403" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 42721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:25.701712 27664 consensus_queue.cc:1035] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430265578104a598de4984140b79af0" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 39089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:25.704098 27671 mvcc.cc:205] Tried to move back new op lower bound from 7333371353909522432 to 7333371353838157824. Current Snapshot: MvccSnapshot[applied={T|T < 7333371353909522432}]
I20260925 22:04:25.705108 27672 mvcc.cc:205] Tried to move back new op lower bound from 7333371353909522432 to 7333371353838157824. Current Snapshot: MvccSnapshot[applied={T|T < 7333371353909522432}]
I20260925 22:04:25.706841 27194 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:25.706936 27194 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:25.707641 27194 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddf0b9be0e70413caacf1f23c4302431: Sending DeleteTablet for 3 replicas of tablet 770b1373ef7d466f91aab7053518b5fe
I20260925 22:04:25.707964 27447 tablet_service.cc:1514] Processing DeleteTablet for tablet 770b1373ef7d466f91aab7053518b5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:25 UTC) from {username='slave'} at 127.0.0.1:57392
I20260925 22:04:25.708012 27578 tablet_service.cc:1514] Processing DeleteTablet for tablet 770b1373ef7d466f91aab7053518b5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:25 UTC) from {username='slave'} at 127.0.0.1:41352
I20260925 22:04:25.708011 27316 tablet_service.cc:1514] Processing DeleteTablet for tablet 770b1373ef7d466f91aab7053518b5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:25 UTC) from {username='slave'} at 127.0.0.1:52324
I20260925 22:04:25.708319 27681 tablet_replica.cc:330] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: stopping tablet replica
I20260925 22:04:25.708423 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27254
I20260925 22:04:25.708446 27681 raft_consensus.cc:2229] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:25.708547 27681 raft_consensus.cc:2258] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:25.708837 27682 tablet_replica.cc:330] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: stopping tablet replica
I20260925 22:04:25.708966 27682 raft_consensus.cc:2229] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:04:25.709096 27682 raft_consensus.cc:2258] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:25.709553 27682 ts_tablet_manager.cc:1905] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:25.710141 27681 ts_tablet_manager.cc:1905] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:25.711814 27682 ts_tablet_manager.cc:1918] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:25.711848 27681 ts_tablet_manager.cc:1918] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:25.711927 27681 log.cc:1196] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/770b1373ef7d466f91aab7053518b5fe
I20260925 22:04:25.712015 27682 log.cc:1196] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/770b1373ef7d466f91aab7053518b5fe
I20260925 22:04:25.712145 27681 ts_tablet_manager.cc:1939] T 770b1373ef7d466f91aab7053518b5fe P 8430265578104a598de4984140b79af0: Deleting consensus metadata
I20260925 22:04:25.712416 27682 ts_tablet_manager.cc:1939] T 770b1373ef7d466f91aab7053518b5fe P 937da6302d6c49b4beba1758dc01c6c9: Deleting consensus metadata
I20260925 22:04:25.712648 27179 catalog_manager.cc:4841] TS 8430265578104a598de4984140b79af0 (127.9.112.66:39089): tablet 770b1373ef7d466f91aab7053518b5fe (table test-table [id=57c21e84832d45af9c88747711309548]) successfully deleted
I20260925 22:04:25.712925 27178 catalog_manager.cc:4841] TS 937da6302d6c49b4beba1758dc01c6c9 (127.9.112.67:44353): tablet 770b1373ef7d466f91aab7053518b5fe (table test-table [id=57c21e84832d45af9c88747711309548]) successfully deleted
W20260925 22:04:25.714447 27180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:42721 (error 108)
I20260925 22:04:25.714689 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27385
I20260925 22:04:25.714913 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:25.714795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:42721 (local address 127.0.0.1:52326)
0925 22:04:25.714845 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:25.714881 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:04:25.715237 27180 catalog_manager.cc:4584] TS e25790dff7dd4df2bc0839f75a6fe403 (127.9.112.65:42721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 770b1373ef7d466f91aab7053518b5fe: Network error: Client connection negotiation failed: client connection to 127.9.112.65:42721: connect: Connection refused (error 111)
I20260925 22:04:25.720572 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27516
I20260925 22:04:25.724853 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27163
2026-09-25T22:04:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (709 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:25 dist-test-slave-794d krb5kdc[27687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:25 dist-test-slave-794d krb5kdc[27687](info): set up 2 sockets
Sep 25 22:04:25 dist-test-slave-794d krb5kdc[27687](info): commencing operation
krb5kdc: starting...
W20260925 22:04:26.700594 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:26 dist-test-slave-794d krb5kdc[27687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:26.836627 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35957
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:26.928089 27700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:26.928421 27700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:26.928454 27700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:26.929808 27700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:26.929857 27700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:26.929869 27700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:26.929881 27700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:26.929893 27700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:26.931923 27700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:26.932235 27700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:26.932518 27700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:26.937849 27708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:26.937974 27705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:26.937996 27706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:26.938357 27700 server_base.cc:1034] running on GCE node
I20260925 22:04:26.938625 27700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:26.938953 27700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:26.940192 27700 hybrid_clock.cc:648] HybridClock initialized: now 1790373866940156 us; error 56 us; skew 500 ppm
Sep 25 22:04:26 dist-test-slave-794d krb5kdc[27687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373866, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:26.943665 27700 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:26.944388 27700 webserver.cc:460] Webserver started at http://127.9.112.126:34311/ using document root <none> and password file <none>
I20260925 22:04:26.944666 27700 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:26.944725 27700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:26.944873 27700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:26.946283 27700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "e7683a711dc44137b7bb68068dd44c09"
format_stamp: "Formatted at 2026-09-25 22:04:26 on dist-test-slave-794d"
server_key: "ec9110e6ae765059f75e86998a5d793a"
server_key_iv: "badec4f26ea8daf7cb74e0bbb9e1e54e"
server_key_version: "encryptionkey@0"
I20260925 22:04:26.946843 27700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "e7683a711dc44137b7bb68068dd44c09"
format_stamp: "Formatted at 2026-09-25 22:04:26 on dist-test-slave-794d"
server_key: "ec9110e6ae765059f75e86998a5d793a"
server_key_iv: "badec4f26ea8daf7cb74e0bbb9e1e54e"
server_key_version: "encryptionkey@0"
I20260925 22:04:26.948630 27700 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 22:04:26.949903 27715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:26.950264 27700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:26.950394 27700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "e7683a711dc44137b7bb68068dd44c09"
format_stamp: "Formatted at 2026-09-25 22:04:26 on dist-test-slave-794d"
server_key: "ec9110e6ae765059f75e86998a5d793a"
server_key_iv: "badec4f26ea8daf7cb74e0bbb9e1e54e"
server_key_version: "encryptionkey@0"
I20260925 22:04:26.950578 27700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:26.972884 27700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:26.976984 27700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:26.977205 27700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:26.982774 27700 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35957
I20260925 22:04:26.982808 27767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35957 every 8 connection(s)
I20260925 22:04:26.983362 27700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:26.984484 27768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:26.987133 27768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: Bootstrap starting.
I20260925 22:04:26.988068 27768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:26.988452 27768 log.cc:824] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:26.989593 27768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: No bootstrap required, opened a new log
I20260925 22:04:26.991518 27768 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7683a711dc44137b7bb68068dd44c09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35957 } }
I20260925 22:04:26.991719 27768 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:26.991743 27768 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7683a711dc44137b7bb68068dd44c09, State: Initialized, Role: FOLLOWER
I20260925 22:04:26.992007 27768 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7683a711dc44137b7bb68068dd44c09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35957 } }
I20260925 22:04:26.992115 27768 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:26.992143 27768 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:26.992197 27768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:26.992878 27768 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7683a711dc44137b7bb68068dd44c09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35957 } }
I20260925 22:04:26.993006 27768 leader_election.cc:302] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7683a711dc44137b7bb68068dd44c09; no voters:
I20260925 22:04:26.993119 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27700
I20260925 22:04:26.993206 27768 leader_election.cc:288] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:26.993234 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:26.993345 27771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:26.993424 9665 external_mini_cluster.cc:1428] Setting key c6bb3acc845c7a73dd74acb3a0775310
I20260925 22:04:26.993542 27768 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:26.993714 27771 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [term 1 LEADER]: Becoming Leader. State: Replica: e7683a711dc44137b7bb68068dd44c09, State: Running, Role: LEADER
I20260925 22:04:26.993947 27771 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7683a711dc44137b7bb68068dd44c09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35957 } }
I20260925 22:04:26.994657 27773 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7683a711dc44137b7bb68068dd44c09. Latest consensus state: current_term: 1 leader_uuid: "e7683a711dc44137b7bb68068dd44c09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7683a711dc44137b7bb68068dd44c09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35957 } } }
I20260925 22:04:26.994796 27773 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:26.994889 27772 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7683a711dc44137b7bb68068dd44c09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7683a711dc44137b7bb68068dd44c09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35957 } } }
I20260925 22:04:26.994972 27772 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09 [sys.catalog]: This master's current role is: LEADER
W20260925 22:04:26.995970 27788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:26.996043 27788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:26.996148 27784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:26.996392 27784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:26.998332 27784 catalog_manager.cc:1347] Generated new cluster ID: 2abb6c0e9523420286c3cd7fc9ce3a6a
I20260925 22:04:26.998411 27784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:26 dist-test-slave-794d krb5kdc[27687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:27.005797 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:26.994040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.9.112.126:35957)
0925 22:04:26.994238 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:26.994252 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:26.994407 (+ 155us) server_negotiation.cc:407] Connection header received
0925 22:04:26.995203 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:26.995213 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:26.995297 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:26.995493 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:26.996392 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:26.997930 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:26.998974 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:26.999475 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:26.999590 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.001972 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:27.001977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:27.001978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:27.001992 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:27.004256 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.004639 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.004642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.004643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.004748 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.004926 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.004927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.004928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.005205 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:27.005380 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.005449 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.005601 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260925 22:04:27.008313 27784 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:27.009178 27784 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:27.020159 27784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: Generated new TSK 0
I20260925 22:04:27.020499 27784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:04:27.062067 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35957
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:27.183648 27794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:27.183969 27794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:27.184000 27794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:27.185448 27794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:27.185503 27794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:27.185534 27794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:27.187300 27794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:27.187601 27794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:27.187846 27794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:27.194186 27799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.194188 27800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.194186 27802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:27.194561 27794 server_base.cc:1034] running on GCE node
I20260925 22:04:27.194959 27794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:27.195290 27794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:27.196440 27794 hybrid_clock.cc:648] HybridClock initialized: now 1790373867196413 us; error 44 us; skew 500 ppm
Sep 25 22:04:27 dist-test-slave-794d krb5kdc[27687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:27.199744 27794 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:04:27.200482 27794 webserver.cc:460] Webserver started at http://127.9.112.65:45391/ using document root <none> and password file <none>
I20260925 22:04:27.200730 27794 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:27.200784 27794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:27.200945 27794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:27.202212 27794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "768aa04334404fb4867779f1859bf036"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "f32b6c0764c2750be0f66775efdd4385"
server_key_iv: "3a3e41c6698595876a110036b640c843"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.202584 27794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "768aa04334404fb4867779f1859bf036"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "f32b6c0764c2750be0f66775efdd4385"
server_key_iv: "3a3e41c6698595876a110036b640c843"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.204265 27794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:27.205487 27809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.205819 27794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:27.205943 27794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "768aa04334404fb4867779f1859bf036"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "f32b6c0764c2750be0f66775efdd4385"
server_key_iv: "3a3e41c6698595876a110036b640c843"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.206054 27794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:27.236449 27794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:27.239434 27794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:27.239755 27794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:27.240090 27794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:27.240516 27794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:27.240559 27794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.240592 27794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:27.240609 27794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.248246 27794 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:34749
I20260925 22:04:27.248374 27922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:34749 every 8 connection(s)
I20260925 22:04:27.248740 27794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:27.249923 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27794
I20260925 22:04:27.250036 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:04:27.250239 9665 external_mini_cluster.cc:1428] Setting key d901462d4ee85f21cadc4d5fc5f769af
Sep 25 22:04:27 dist-test-slave-794d krb5kdc[27687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
I20260925 22:04:27.261667 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.249618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:53599 (local address 127.9.112.126:35957)
0925 22:04:27.249756 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.249759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.250693 (+ 934us) server_negotiation.cc:407] Connection header received
0925 22:04:27.251399 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.251401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.251420 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.251531 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:27.252681 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.253292 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.254627 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.254956 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.255039 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.258438 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:27.258441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:27.258441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:27.258450 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:27.260047 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.260597 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.260598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.260599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.260654 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.260936 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.260937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.260938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.261059 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:27.261116 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.261416 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.261570 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 22:04:27.261821 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.249579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35957 (local address 127.9.112.65:53599)
0925 22:04:27.250493 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.250554 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.251246 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.251600 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.251604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.252265 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:27.252567 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.252572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.253409 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.253410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.254518 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.254521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.255078 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.255079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.255213 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.255913 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:27.255923 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:27.258315 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:27.260145 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:27.260157 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:27.260161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:27.260523 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:27.260716 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:27.260717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:27.260717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:27.260868 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:27.261118 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:27.261125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:27.261281 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.261372 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.261565 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":154,"threads_started":1}
I20260925 22:04:27.262555 27923 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35957
I20260925 22:04:27.262671 27923 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:27.262923 27923 heartbeater.cc:511] Master 127.9.112.126:35957 requested a full tablet report, sending...
I20260925 22:04:27.263566 27732 ts_manager.cc:194] Registered new tserver with Master: 768aa04334404fb4867779f1859bf036 (127.9.112.65:34749)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
I20260925 22:04:27.264281 27732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:53599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:04:27.285704 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35957
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:27.381896 27928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:27.382189 27928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:27.382220 27928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:27.384145 27928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:27.384234 27928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:27.384271 27928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:27.386073 27928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:27.386338 27928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:27.386582 27928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:27.392462 27933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.392676 27934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:27.392724 27928 server_base.cc:1034] running on GCE node
W20260925 22:04:27.392771 27936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:27.393018 27928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:27.393282 27928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:27.394651 27928 hybrid_clock.cc:648] HybridClock initialized: now 1790373867394628 us; error 29 us; skew 500 ppm
Sep 25 22:04:27 dist-test-slave-794d krb5kdc[27687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:27.397809 27928 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:04:27.398342 27928 webserver.cc:460] Webserver started at http://127.9.112.66:33413/ using document root <none> and password file <none>
I20260925 22:04:27.398581 27928 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:27.398633 27928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:27.398773 27928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:27.399967 27928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "45488b49eca3427ba866fce40942750b"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "5547d1752fd4cf97bc0ccee0a16fca69"
server_key_iv: "e4db52fe9fba5f4af94f5c5dcbc73e40"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.400382 27928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "45488b49eca3427ba866fce40942750b"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "5547d1752fd4cf97bc0ccee0a16fca69"
server_key_iv: "e4db52fe9fba5f4af94f5c5dcbc73e40"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.401834 27928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:27.402767 27943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.403018 27928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:27.403119 27928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "45488b49eca3427ba866fce40942750b"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "5547d1752fd4cf97bc0ccee0a16fca69"
server_key_iv: "e4db52fe9fba5f4af94f5c5dcbc73e40"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.403203 27928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:27.416317 27928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:27.418874 27928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:27.419039 27928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:27.419317 27928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:27.419691 27928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:27.419729 27928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.419762 27928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:27.419783 27928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.426792 27928 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:36713
I20260925 22:04:27.426846 28056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:36713 every 8 connection(s)
I20260925 22:04:27.427206 27928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:27.432153 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 27928
I20260925 22:04:27.432279 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:04:27.432571 9665 external_mini_cluster.cc:1428] Setting key 7f6dfb5f05fee5bd9626e4ca8b45e043
Sep 25 22:04:27 dist-test-slave-794d krb5kdc[27687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:27.438894 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.428109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:42661 (local address 127.9.112.126:35957)
0925 22:04:27.428172 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.428174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.428400 (+ 226us) server_negotiation.cc:407] Connection header received
0925 22:04:27.429060 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.429061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.429073 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.429140 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:27.430459 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.431099 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.432243 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.432566 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.432646 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.435513 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:27.435515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:27.435516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:27.435525 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:27.437255 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.437702 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.437703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.437704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.437769 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.438130 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.438130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.438131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.438284 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:27.438358 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.438670 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.438785 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:04:27.438980 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.428031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35957 (local address 127.9.112.66:42661)
0925 22:04:27.428205 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.428254 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.428877 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.429242 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.429246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.429931 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:27.430318 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.430324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.431207 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.431208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.432138 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.432142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.432667 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.432668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.432813 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.433531 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:27.433543 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:27.435411 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:27.437374 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:27.437382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:27.437385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:27.437630 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:27.437867 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:27.437869 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:27.437869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:27.438057 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:27.438351 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:27.438355 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:27.438528 (+ 173us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.438623 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.438778 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260925 22:04:27.439661 28057 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35957
I20260925 22:04:27.439780 28057 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:27.440050 28057 heartbeater.cc:511] Master 127.9.112.126:35957 requested a full tablet report, sending...
I20260925 22:04:27.440624 27732 ts_manager.cc:194] Registered new tserver with Master: 45488b49eca3427ba866fce40942750b (127.9.112.66:36713)
I20260925 22:04:27.441210 27732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:42661
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:04:27.464763 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35957
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:27.557988 28062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:27.558233 28062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:27.558257 28062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:27.559456 28062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:27.559499 28062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:27.559523 28062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:27.561192 28062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:27.561439 28062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:27.561662 28062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:27.567093 28067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.567094 28068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.567093 28070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:27.567365 28062 server_base.cc:1034] running on GCE node
I20260925 22:04:27.567651 28062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:27.567904 28062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:27.569067 28062 hybrid_clock.cc:648] HybridClock initialized: now 1790373867568991 us; error 87 us; skew 500 ppm
Sep 25 22:04:27 dist-test-slave-794d krb5kdc[27687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:27.572136 28062 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:04:27.572712 28062 webserver.cc:460] Webserver started at http://127.9.112.67:35495/ using document root <none> and password file <none>
I20260925 22:04:27.572968 28062 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:27.573024 28062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:27.573158 28062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:27.574419 28062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "c3dcd14cc8af4a62b3080f1726876634"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "83e042a584ec1cea5279a634d6be9ba3"
server_key_iv: "fd71faba0e7d44184ef28a15c9599dd0"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.574772 28062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "c3dcd14cc8af4a62b3080f1726876634"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "83e042a584ec1cea5279a634d6be9ba3"
server_key_iv: "fd71faba0e7d44184ef28a15c9599dd0"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.576159 28062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:27.577033 28077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.577220 28062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:04:27.577308 28062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "c3dcd14cc8af4a62b3080f1726876634"
format_stamp: "Formatted at 2026-09-25 22:04:27 on dist-test-slave-794d"
server_key: "83e042a584ec1cea5279a634d6be9ba3"
server_key_iv: "fd71faba0e7d44184ef28a15c9599dd0"
server_key_version: "encryptionkey@0"
I20260925 22:04:27.577412 28062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:27.591768 28062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:27.594521 28062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:27.594700 28062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:27.595034 28062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:27.595397 28062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:27.595440 28062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.595475 28062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:27.595491 28062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:27.602190 28062 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:38095
I20260925 22:04:27.602231 28190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:38095 every 8 connection(s)
I20260925 22:04:27.602588 28062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
Sep 25 22:04:27 dist-test-slave-794d krb5kdc[27687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:27.611073 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28062
I20260925 22:04:27.611173 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:04:27.611351 9665 external_mini_cluster.cc:1428] Setting key a9ca688faec636c078538c1efc94b189
I20260925 22:04:27.613689 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.603479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:36975 (local address 127.9.112.126:35957)
0925 22:04:27.603539 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.603541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.603747 (+ 206us) server_negotiation.cc:407] Connection header received
0925 22:04:27.604463 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.604464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.604477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.604531 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:27.605611 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.606218 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.607467 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.607743 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.607803 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.610485 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:27.610487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:27.610487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:27.610497 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:27.612173 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.612674 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.612676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.612676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.612738 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.613051 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.613052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.613052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.613181 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:27.613248 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.613534 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.613613 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:04:27.613854 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.603419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35957 (local address 127.9.112.67:36975)
0925 22:04:27.603592 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.603639 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.604299 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.604610 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.604615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.605256 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:27.605497 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.605502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.606336 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.606337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.607371 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.607374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.607839 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.607840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.607990 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.608633 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:27.608643 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:27.610292 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:27.612268 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:27.612277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:27.612280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:27.612590 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:27.612800 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:27.612801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:27.612801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:27.612903 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:27.613242 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:27.613246 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:27.613381 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.613465 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.613637 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260925 22:04:27.614490 28191 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35957
I20260925 22:04:27.614607 28191 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:27.614807 28191 heartbeater.cc:511] Master 127.9.112.126:35957 requested a full tablet report, sending...
I20260925 22:04:27.615216 27732 ts_manager.cc:194] Registered new tserver with Master: c3dcd14cc8af4a62b3080f1726876634 (127.9.112.67:38095)
I20260925 22:04:27.615629 27732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:36975
I20260925 22:04:27.623283 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:04:27.631609 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.624872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.9.112.126:35957)
0925 22:04:27.624953 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.624955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.625049 (+ 94us) server_negotiation.cc:407] Connection header received
0925 22:04:27.625105 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.625106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.625116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.625168 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:27.625756 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.626335 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.627223 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.627427 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.627491 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.628952 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:27.628954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:27.628954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:27.628960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:27.630197 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.630678 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.630679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.630680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.630754 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:27.631065 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:27.631066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:27.631066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:27.631218 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:27.631313 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.631444 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.631507 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:04:27.633620 27732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:04:27.635682 27732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:04:27.643638 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.638830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.9.112.66:36713)
0925 22:04:27.639165 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.639170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.639181 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:27.639228 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.639230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.639245 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.639331 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:27.639707 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.640649 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.642168 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.643001 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.643082 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.643170 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.643473 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.643529 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":169,"threads_started":1}
I20260925 22:04:27.643631 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.638852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34749 (local address 127.0.0.1:51414)
0925 22:04:27.639107 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.639116 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.639159 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.639865 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.639866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.639868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:27.639905 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.639906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.641464 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.641465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.642369 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.642380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.643362 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.643363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.643434 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.643438 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.643483 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.643531 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":151,"threads_started":1}
I20260925 22:04:27.643754 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.638734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.9.112.67:38095)
0925 22:04:27.639393 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.639398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.639409 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:27.639461 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.639464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.639488 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.639602 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:27.639723 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.641065 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.642394 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.643115 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.643234 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.643310 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.643604 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.643668 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":103,"threads_started":1}
I20260925 22:04:27.643635 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.638619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:36713 (local address 127.0.0.1:45816)
0925 22:04:27.639033 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.639044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.639113 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.639451 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.639454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.639457 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:27.639551 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.639553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.640747 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.640748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.642052 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.642054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.643160 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.643161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.643290 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.643310 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.643400 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.643493 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":37,"thread_start_us":109,"threads_started":1}
I20260925 22:04:27.644181 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.639207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.9.112.65:34749)
0925 22:04:27.639701 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.639705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.639715 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:04:27.639756 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.639759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.639777 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.639860 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:27.639991 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.641316 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.642502 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.643246 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.643348 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.643418 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.644022 (+ 604us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.644091 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":109,"threads_started":1}
I20260925 22:04:27.643882 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.638611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:38095 (local address 127.0.0.1:43246)
0925 22:04:27.638837 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.638888 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.638978 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.639601 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.639602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.639604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:27.639657 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.639658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.641227 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.641228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.642305 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.642307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.643246 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.643247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.643338 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.643343 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.643392 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.643490 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":19,"thread_start_us":91,"threads_started":1}
I20260925 22:04:27.644977 27857 tablet_service.cc:1467] Processing CreateTablet for tablet 3589d45e20b04b00941c8a971845eb7b (DEFAULT_TABLE table=test-table [id=c541be0c3eb243f890015d97988fa68d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:27.644934 27991 tablet_service.cc:1467] Processing CreateTablet for tablet 3589d45e20b04b00941c8a971845eb7b (DEFAULT_TABLE table=test-table [id=c541be0c3eb243f890015d97988fa68d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:27.645000 28125 tablet_service.cc:1467] Processing CreateTablet for tablet 3589d45e20b04b00941c8a971845eb7b (DEFAULT_TABLE table=test-table [id=c541be0c3eb243f890015d97988fa68d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:27.645339 27857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3589d45e20b04b00941c8a971845eb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:27.645341 28125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3589d45e20b04b00941c8a971845eb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:27.645350 27991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3589d45e20b04b00941c8a971845eb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:27.649044 28208 tablet_bootstrap.cc:492] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: Bootstrap starting.
I20260925 22:04:27.649525 28209 tablet_bootstrap.cc:492] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Bootstrap starting.
I20260925 22:04:27.649801 28210 tablet_bootstrap.cc:492] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Bootstrap starting.
I20260925 22:04:27.650090 28208 tablet_bootstrap.cc:654] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:27.650266 28209 tablet_bootstrap.cc:654] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:27.650512 28210 tablet_bootstrap.cc:654] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:27.650804 28208 log.cc:824] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:27.650851 28209 log.cc:824] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:27.651049 28210 log.cc:824] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:27.651713 28208 tablet_bootstrap.cc:492] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: No bootstrap required, opened a new log
I20260925 22:04:27.651809 28208 ts_tablet_manager.cc:1397] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:04:27.651942 28210 tablet_bootstrap.cc:492] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: No bootstrap required, opened a new log
I20260925 22:04:27.652010 28210 ts_tablet_manager.cc:1397] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:27.652249 28209 tablet_bootstrap.cc:492] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: No bootstrap required, opened a new log
I20260925 22:04:27.652318 28209 ts_tablet_manager.cc:1397] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 22:04:27.653945 28209 raft_consensus.cc:348] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.654137 28208 raft_consensus.cc:348] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.654124 28210 raft_consensus.cc:348] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.654289 28209 raft_consensus.cc:374] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:27.654302 28210 raft_consensus.cc:374] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:27.654302 28208 raft_consensus.cc:374] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:27.654332 28209 raft_consensus.cc:729] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45488b49eca3427ba866fce40942750b, State: Initialized, Role: FOLLOWER
I20260925 22:04:27.654335 28210 raft_consensus.cc:729] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3dcd14cc8af4a62b3080f1726876634, State: Initialized, Role: FOLLOWER
I20260925 22:04:27.654335 28208 raft_consensus.cc:729] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768aa04334404fb4867779f1859bf036, State: Initialized, Role: FOLLOWER
I20260925 22:04:27.654510 28210 consensus_queue.cc:260] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.654505 28208 consensus_queue.cc:260] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.654516 28209 consensus_queue.cc:260] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.654856 28209 ts_tablet_manager.cc:1428] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:27.654855 28208 ts_tablet_manager.cc:1428] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:04:27.654855 28210 ts_tablet_manager.cc:1428] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 22:04:27.654899 28057 heartbeater.cc:503] Master 127.9.112.126:35957 was elected leader, sending a full tablet report...
I20260925 22:04:27.655001 27923 heartbeater.cc:503] Master 127.9.112.126:35957 was elected leader, sending a full tablet report...
I20260925 22:04:27.655026 28191 heartbeater.cc:503] Master 127.9.112.126:35957 was elected leader, sending a full tablet report...
W20260925 22:04:27.677879 28058 tablet.cc:2369] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:04:27.749644 27924 tablet.cc:2369] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:04:27.799247 28216 raft_consensus.cc:482] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:27.799424 28216 raft_consensus.cc:504] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.799957 28216 leader_election.cc:288] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45488b49eca3427ba866fce40942750b (127.9.112.66:36713), c3dcd14cc8af4a62b3080f1726876634 (127.9.112.67:38095)
I20260925 22:04:27.804939 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.800401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:46695 (local address 127.9.112.66:36713)
0925 22:04:27.800454 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.800456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.800524 (+ 68us) server_negotiation.cc:407] Connection header received
0925 22:04:27.800614 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.800615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.800627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.800713 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:27.801183 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.802033 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.803753 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.804450 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.804519 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.804528 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.804791 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.804851 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:04:27.804939 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.800400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:40029 (local address 127.9.112.67:38095)
0925 22:04:27.800454 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.800456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.800585 (+ 129us) server_negotiation.cc:407] Connection header received
0925 22:04:27.800639 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.800640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.800650 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.800713 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:27.801186 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.802028 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.803754 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.804410 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.804511 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.804525 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.804791 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.804848 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 22:04:27.804949 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.800229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:38095 (local address 127.9.112.65:40029)
0925 22:04:27.800514 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.800523 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.800580 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.800838 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.800840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.800846 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:27.801032 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.801035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.802166 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.802168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.803634 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.803639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.804551 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.804552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.804659 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.804671 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.804742 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.804803 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":105,"thread_start_us":98,"threads_started":1}
I20260925 22:04:27.804951 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.800229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:36713 (local address 127.9.112.65:46695)
0925 22:04:27.800438 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.800451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:27.800531 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:27.800838 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:27.800840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:27.800846 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:27.801009 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.801012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.802166 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.802168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.803635 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:27.803639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.804549 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.804551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:27.804659 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.804670 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 22:04:27.804742 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 22:04:27.804803 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":116,"thread_start_us":133,"threads_started":1}
I20260925 22:04:27.805477 28145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589d45e20b04b00941c8a971845eb7b" candidate_uuid: "768aa04334404fb4867779f1859bf036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3dcd14cc8af4a62b3080f1726876634" is_pre_election: true
I20260925 22:04:27.805488 28011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589d45e20b04b00941c8a971845eb7b" candidate_uuid: "768aa04334404fb4867779f1859bf036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45488b49eca3427ba866fce40942750b" is_pre_election: true
I20260925 22:04:27.805680 28145 raft_consensus.cc:2454] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768aa04334404fb4867779f1859bf036 in term 0.
I20260925 22:04:27.805681 28011 raft_consensus.cc:2454] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768aa04334404fb4867779f1859bf036 in term 0.
I20260925 22:04:27.805958 27810 leader_election.cc:302] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768aa04334404fb4867779f1859bf036, c3dcd14cc8af4a62b3080f1726876634; no voters:
I20260925 22:04:27.806136 28216 raft_consensus.cc:2781] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:04:27.806200 28216 raft_consensus.cc:482] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:04:27.806218 28216 raft_consensus.cc:3037] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:27.807201 28216 raft_consensus.cc:504] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.807348 28216 leader_election.cc:288] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [CANDIDATE]: Term 1 election: Requested vote from peers 45488b49eca3427ba866fce40942750b (127.9.112.66:36713), c3dcd14cc8af4a62b3080f1726876634 (127.9.112.67:38095)
I20260925 22:04:27.807528 28011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589d45e20b04b00941c8a971845eb7b" candidate_uuid: "768aa04334404fb4867779f1859bf036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45488b49eca3427ba866fce40942750b"
I20260925 22:04:27.807570 28145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589d45e20b04b00941c8a971845eb7b" candidate_uuid: "768aa04334404fb4867779f1859bf036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3dcd14cc8af4a62b3080f1726876634"
I20260925 22:04:27.807621 28011 raft_consensus.cc:3037] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:27.807646 28145 raft_consensus.cc:3037] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:27.808496 28145 raft_consensus.cc:2454] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768aa04334404fb4867779f1859bf036 in term 1.
I20260925 22:04:27.808614 28011 raft_consensus.cc:2454] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768aa04334404fb4867779f1859bf036 in term 1.
I20260925 22:04:27.808661 27810 leader_election.cc:302] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768aa04334404fb4867779f1859bf036, c3dcd14cc8af4a62b3080f1726876634; no voters:
I20260925 22:04:27.808763 28216 raft_consensus.cc:2781] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:27.809043 28216 raft_consensus.cc:686] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [term 1 LEADER]: Becoming Leader. State: Replica: 768aa04334404fb4867779f1859bf036, State: Running, Role: LEADER
I20260925 22:04:27.809157 28216 consensus_queue.cc:237] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } }
I20260925 22:04:27.810328 27731 catalog_manager.cc:5495] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 reported cstate change: term changed from 0 to 1, leader changed from <none> to 768aa04334404fb4867779f1859bf036 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "768aa04334404fb4867779f1859bf036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768aa04334404fb4867779f1859bf036" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 34749 } health_report { overall_health: HEALTHY } } }
I20260925 22:04:27.831020 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.828187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.9.112.65:34749)
0925 22:04:27.828254 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:27.828256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:27.828263 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:04:27.828308 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:27.828309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:27.828334 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:27.828439 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:27.828595 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.829134 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.830088 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:27.830293 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:27.830352 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:27.830640 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:27.830756 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:27.830857 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:27.830908 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 22:04:27.830921 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 22:04:27.833770 28011 raft_consensus.cc:1261] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 1 FOLLOWER]: Refusing update from remote peer 768aa04334404fb4867779f1859bf036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:27.833770 28145 raft_consensus.cc:1261] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 1 FOLLOWER]: Refusing update from remote peer 768aa04334404fb4867779f1859bf036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:27.834273 28216 consensus_queue.cc:1035] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45488b49eca3427ba866fce40942750b" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 36713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:27.834457 28219 consensus_queue.cc:1035] T 3589d45e20b04b00941c8a971845eb7b P 768aa04334404fb4867779f1859bf036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3dcd14cc8af4a62b3080f1726876634" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:27.837309 28224 mvcc.cc:205] Tried to move back new op lower bound from 7333371362642493440 to 7333371362547089408. Current Snapshot: MvccSnapshot[applied={T|T < 7333371362642493440}]
I20260925 22:04:27.837759 28223 mvcc.cc:205] Tried to move back new op lower bound from 7333371362642493440 to 7333371362547089408. Current Snapshot: MvccSnapshot[applied={T|T < 7333371362642493440}]
I20260925 22:04:27.837855 28222 mvcc.cc:205] Tried to move back new op lower bound from 7333371362642493440 to 7333371362547089408. Current Snapshot: MvccSnapshot[applied={T|T < 7333371362642493440 or (T in {7333371362642493440})}]
I20260925 22:04:27.840232 27730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:27.840375 27730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:27.841357 27730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7683a711dc44137b7bb68068dd44c09: Sending DeleteTablet for 3 replicas of tablet 3589d45e20b04b00941c8a971845eb7b
I20260925 22:04:27.841701 28125 tablet_service.cc:1514] Processing DeleteTablet for tablet 3589d45e20b04b00941c8a971845eb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:27 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:43246
I20260925 22:04:27.841756 27857 tablet_service.cc:1514] Processing DeleteTablet for tablet 3589d45e20b04b00941c8a971845eb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:27 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:51414
I20260925 22:04:27.841786 27991 tablet_service.cc:1514] Processing DeleteTablet for tablet 3589d45e20b04b00941c8a971845eb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:27 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:45816
I20260925 22:04:27.842048 28233 tablet_replica.cc:330] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: stopping tablet replica
I20260925 22:04:27.842187 28233 raft_consensus.cc:2229] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:27.842221 28234 tablet_replica.cc:330] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: stopping tablet replica
I20260925 22:04:27.842342 28233 raft_consensus.cc:2258] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:27.842432 28234 raft_consensus.cc:2229] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:27.842578 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27794
I20260925 22:04:27.842619 28234 raft_consensus.cc:2258] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:27.843000 28233 ts_tablet_manager.cc:1905] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:27.843789 28234 ts_tablet_manager.cc:1905] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:27.845271 28233 ts_tablet_manager.cc:1918] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:27.845376 28233 log.cc:1196] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/3589d45e20b04b00941c8a971845eb7b
I20260925 22:04:27.845674 28233 ts_tablet_manager.cc:1939] T 3589d45e20b04b00941c8a971845eb7b P c3dcd14cc8af4a62b3080f1726876634: Deleting consensus metadata
I20260925 22:04:27.845753 28234 ts_tablet_manager.cc:1918] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:27.845826 28234 log.cc:1196] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/3589d45e20b04b00941c8a971845eb7b
I20260925 22:04:27.846107 28234 ts_tablet_manager.cc:1939] T 3589d45e20b04b00941c8a971845eb7b P 45488b49eca3427ba866fce40942750b: Deleting consensus metadata
I20260925 22:04:27.846143 27716 catalog_manager.cc:4841] TS c3dcd14cc8af4a62b3080f1726876634 (127.9.112.67:38095): tablet 3589d45e20b04b00941c8a971845eb7b (table test-table [id=c541be0c3eb243f890015d97988fa68d]) successfully deleted
I20260925 22:04:27.846519 27717 catalog_manager.cc:4841] TS 45488b49eca3427ba866fce40942750b (127.9.112.66:36713): tablet 3589d45e20b04b00941c8a971845eb7b (table test-table [id=c541be0c3eb243f890015d97988fa68d]) successfully deleted
W20260925 22:04:27.849525 27716 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:34749 (error 108)
I20260925 22:04:27.849781 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27928
I20260925 22:04:27.850641 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:27.850490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:34749 (local address 127.0.0.1:51428)
0925 22:04:27.850551 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:27.850598 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:34749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 22:04:27.850764 27716 catalog_manager.cc:4584] TS 768aa04334404fb4867779f1859bf036 (127.9.112.65:34749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3589d45e20b04b00941c8a971845eb7b: Network error: Client connection negotiation failed: client connection to 127.9.112.65:34749: connect: Connection refused (error 111)
I20260925 22:04:27.855355 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28062
I20260925 22:04:27.862551 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 27700
2026-09-25T22:04:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2139 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T22:04:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:27Z Disabled control of system clock
I20260925 22:04:27.891865 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40525
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:43897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:27.987635 28239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:27.987926 28239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:27.987950 28239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:27.989351 28239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:27.989414 28239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:27.989427 28239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:27.989445 28239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:27.989456 28239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:27.990974 28239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:27.991220 28239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:27.991514 28239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:27.997103 28245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.997088 28244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:27.997087 28247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:27.997479 28239 server_base.cc:1034] running on GCE node
I20260925 22:04:27.997784 28239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:27.998136 28239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:27.999310 28239 hybrid_clock.cc:648] HybridClock initialized: now 1790373867999290 us; error 39 us; skew 500 ppm
I20260925 22:04:28.007452 28239 webserver.cc:460] Webserver started at http://127.9.112.126:39749/ using document root <none> and password file <none>
I20260925 22:04:28.007768 28239 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:28.007827 28239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:28.008013 28239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:28.009361 28239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "6c5f0ceacae24bfc893f5ed995a12036"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "7d04870c4b1d716f87f41728a852f23c"
server_key_iv: "4fa369bd1c763ca2de1706ccdd163a9f"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.009743 28239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "6c5f0ceacae24bfc893f5ed995a12036"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "7d04870c4b1d716f87f41728a852f23c"
server_key_iv: "4fa369bd1c763ca2de1706ccdd163a9f"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.011386 28239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:04:28.012329 28253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.012565 28239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:28.012661 28239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "6c5f0ceacae24bfc893f5ed995a12036"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "7d04870c4b1d716f87f41728a852f23c"
server_key_iv: "4fa369bd1c763ca2de1706ccdd163a9f"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.012758 28239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:28.025117 28239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:28.025506 28239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:28.025647 28239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:28.030321 28239 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40525
I20260925 22:04:28.030369 28305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40525 every 8 connection(s)
I20260925 22:04:28.030762 28239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:28.031955 28306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:28.034804 28306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036: Bootstrap starting.
I20260925 22:04:28.035470 28306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:28.035777 28306 log.cc:824] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:28.036643 28306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036: No bootstrap required, opened a new log
I20260925 22:04:28.037186 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28239
I20260925 22:04:28.037292 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:28.037505 9665 external_mini_cluster.cc:1428] Setting key 572ead2661375b45adde3d028278d816
I20260925 22:04:28.038787 28306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5f0ceacae24bfc893f5ed995a12036" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40525 } }
I20260925 22:04:28.038961 28306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:28.039042 28306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5f0ceacae24bfc893f5ed995a12036, State: Initialized, Role: FOLLOWER
I20260925 22:04:28.039214 28306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5f0ceacae24bfc893f5ed995a12036" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40525 } }
I20260925 22:04:28.039299 28306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:28.039333 28306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:28.039383 28306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:28.040009 28306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5f0ceacae24bfc893f5ed995a12036" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40525 } }
I20260925 22:04:28.040113 28306 leader_election.cc:302] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c5f0ceacae24bfc893f5ed995a12036; no voters:
I20260925 22:04:28.040299 28306 leader_election.cc:288] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:28.040377 28311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:28.040524 28311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [term 1 LEADER]: Becoming Leader. State: Replica: 6c5f0ceacae24bfc893f5ed995a12036, State: Running, Role: LEADER
I20260925 22:04:28.040627 28306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:28.040668 28311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5f0ceacae24bfc893f5ed995a12036" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40525 } }
I20260925 22:04:28.041087 28311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c5f0ceacae24bfc893f5ed995a12036. Latest consensus state: current_term: 1 leader_uuid: "6c5f0ceacae24bfc893f5ed995a12036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5f0ceacae24bfc893f5ed995a12036" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40525 } } }
I20260925 22:04:28.041169 28311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:28.041353 28313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c5f0ceacae24bfc893f5ed995a12036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c5f0ceacae24bfc893f5ed995a12036" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40525 } } }
I20260925 22:04:28.041431 28313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:28.041705 28319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:28.042538 28319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:28.044512 28319 catalog_manager.cc:1347] Generated new cluster ID: 0e38c567b9f64149aaf403fdba8c4535
I20260925 22:04:28.044574 28319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:28.044760 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.038155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.9.112.126:40525)
0925 22:04:28.038409 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.038418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.038445 (+ 27us) server_negotiation.cc:407] Connection header received
0925 22:04:28.038780 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.038790 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.038864 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.039094 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:28.039324 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.040990 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.042162 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.042960 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.043103 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.043921 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:28.043926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:28.043927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:28.044181 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:28.044229 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:28.044246 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:28.044316 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.044433 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.044526 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260925 22:04:28.052258 28319 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:28.053085 28319 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:28.065299 28319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036: Generated new TSK 0
I20260925 22:04:28.065580 28319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:28.076836 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40525
--builtin_ntp_servers=127.9.112.84:43897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:28.172624 28330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:28.172879 28330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:28.172911 28330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:28.174170 28330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:28.174211 28330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:28.174247 28330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:28.175676 28330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:28.175935 28330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:28.176259 28330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:28.182838 28335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:28.182832 28336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:28.182997 28338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:28.183218 28330 server_base.cc:1034] running on GCE node
I20260925 22:04:28.183518 28330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:28.183801 28330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:28.184973 28330 hybrid_clock.cc:648] HybridClock initialized: now 1790373868184945 us; error 43 us; skew 500 ppm
I20260925 22:04:28.186386 28330 webserver.cc:460] Webserver started at http://127.9.112.65:42725/ using document root <none> and password file <none>
I20260925 22:04:28.186849 28330 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:28.186913 28330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:28.187057 28330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:28.188309 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "7089d76792ec4b68973881fc454387d1"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "85e55c56b1b28d31b2f08635d9f175d1"
server_key_iv: "d0df6b18fc545499417accf92231bb06"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.188723 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "7089d76792ec4b68973881fc454387d1"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "85e55c56b1b28d31b2f08635d9f175d1"
server_key_iv: "d0df6b18fc545499417accf92231bb06"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.190143 28330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:28.190995 28344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.191165 28330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:28.191248 28330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "7089d76792ec4b68973881fc454387d1"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "85e55c56b1b28d31b2f08635d9f175d1"
server_key_iv: "d0df6b18fc545499417accf92231bb06"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.191320 28330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:28.209640 28330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:28.210069 28330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:28.210227 28330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:28.210516 28330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:28.210896 28330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:28.210932 28330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.210966 28330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:28.210990 28330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.217864 28330 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:44905
I20260925 22:04:28.217911 28457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:44905 every 8 connection(s)
I20260925 22:04:28.218293 28330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:28.223212 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28330
I20260925 22:04:28.223330 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:04:28.223528 9665 external_mini_cluster.cc:1428] Setting key afcf767c9b98a71b98daac1ff3db5ffb
I20260925 22:04:28.224315 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.219192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:33147 (local address 127.9.112.126:40525)
0925 22:04:28.219283 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.219285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.219446 (+ 161us) server_negotiation.cc:407] Connection header received
0925 22:04:28.220096 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.220097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.220112 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.220165 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:28.220916 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.221463 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.222686 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.222898 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.222982 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.224032 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:28.224034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:28.224034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:28.224039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:28.224059 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:28.224063 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:28.224110 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.224187 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.224225 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 22:04:28.224610 28460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.219105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40525 (local address 127.9.112.65:33147)
0925 22:04:28.219304 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.219348 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.219972 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.220238 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.220242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.220251 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:28.220847 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.220852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.221567 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.221568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.222485 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.222488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.223084 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.223085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.223229 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.223888 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:28.223889 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:28.223922 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:28.223950 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:28.224116 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:28.224123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:28.224126 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.224187 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.224317 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260925 22:04:28.224610 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40525
--builtin_ntp_servers=127.9.112.84:43897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:04:28.225215 28458 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40525
I20260925 22:04:28.225358 28458 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:28.225594 28458 heartbeater.cc:511] Master 127.9.112.126:40525 requested a full tablet report, sending...
I20260925 22:04:28.226246 28270 ts_manager.cc:194] Registered new tserver with Master: 7089d76792ec4b68973881fc454387d1 (127.9.112.65:44905)
I20260925 22:04:28.227144 28270 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.65:33147
W20260925 22:04:28.318236 28461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:28.318500 28461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:28.318532 28461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:28.319784 28461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:28.319826 28461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:28.319860 28461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:28.321346 28461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:28.321625 28461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:28.321964 28461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:28.327454 28469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:28.327446 28467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:28.327644 28466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:28.327811 28461 server_base.cc:1034] running on GCE node
I20260925 22:04:28.328051 28461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:28.328307 28461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:28.329500 28461 hybrid_clock.cc:648] HybridClock initialized: now 1790373868329466 us; error 52 us; skew 500 ppm
I20260925 22:04:28.331084 28461 webserver.cc:460] Webserver started at http://127.9.112.66:35777/ using document root <none> and password file <none>
I20260925 22:04:28.331337 28461 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:28.331385 28461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:28.331526 28461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:28.332828 28461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "bb4fc08987d04bb5a44b08470966a504"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "fea1f0bb606527bfd3e0dc1017451db6"
server_key_iv: "3b72f713855695e42bdc4e4fee5d67e2"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.333200 28461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "bb4fc08987d04bb5a44b08470966a504"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "fea1f0bb606527bfd3e0dc1017451db6"
server_key_iv: "3b72f713855695e42bdc4e4fee5d67e2"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.334564 28461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:28.335410 28475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.335546 28461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:04:28.335630 28461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "bb4fc08987d04bb5a44b08470966a504"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "fea1f0bb606527bfd3e0dc1017451db6"
server_key_iv: "3b72f713855695e42bdc4e4fee5d67e2"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.335698 28461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:28.353734 28461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:28.354159 28461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:28.354311 28461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:28.354586 28461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:28.355052 28461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:28.355118 28461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.355156 28461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:28.355182 28461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.362223 28461 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:40435
I20260925 22:04:28.362273 28588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:40435 every 8 connection(s)
I20260925 22:04:28.362660 28461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:28.368630 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.363545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:59971 (local address 127.9.112.126:40525)
0925 22:04:28.363611 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.363613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.363825 (+ 212us) server_negotiation.cc:407] Connection header received
0925 22:04:28.364463 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.364465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.364481 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.364564 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:28.365326 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.365944 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.367039 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.367257 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.367315 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.368250 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:28.368251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:28.368252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:28.368257 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:28.368275 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:28.368280 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:28.368326 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.368508 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.368555 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:04:28.368816 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.363491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40525 (local address 127.9.112.66:59971)
0925 22:04:28.363691 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.363738 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.364295 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.364634 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.364638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.364648 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:28.365211 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.365215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.366044 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.366045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.366947 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.366950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.367368 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.367369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.367499 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.368093 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:28.368094 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:28.368127 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:28.368152 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:28.368410 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:28.368417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:28.368421 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.368500 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.368598 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260925 22:04:28.369342 28589 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40525
I20260925 22:04:28.369446 28589 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:28.369654 28589 heartbeater.cc:511] Master 127.9.112.126:40525 requested a full tablet report, sending...
I20260925 22:04:28.370090 28270 ts_manager.cc:194] Registered new tserver with Master: bb4fc08987d04bb5a44b08470966a504 (127.9.112.66:40435)
I20260925 22:04:28.370499 28270 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.66:59971
I20260925 22:04:28.371539 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28461
I20260925 22:04:28.371636 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:04:28.371835 9665 external_mini_cluster.cc:1428] Setting key d48bda914a4f0d95f9caf63a3d6f379c
I20260925 22:04:28.372822 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40525
--builtin_ntp_servers=127.9.112.84:43897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:28.466049 28592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:28.466295 28592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:28.466325 28592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:28.467552 28592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:28.467597 28592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:28.467631 28592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:28.469100 28592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:43897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:28.469370 28592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:28.469686 28592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:28.475646 28600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:28.475795 28592 server_base.cc:1034] running on GCE node
W20260925 22:04:28.475651 28598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:28.475651 28597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:28.476274 28592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:28.476583 28592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:28.477743 28592 hybrid_clock.cc:648] HybridClock initialized: now 1790373868477724 us; error 30 us; skew 500 ppm
I20260925 22:04:28.479293 28592 webserver.cc:460] Webserver started at http://127.9.112.67:39407/ using document root <none> and password file <none>
I20260925 22:04:28.479562 28592 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:28.479616 28592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:28.479751 28592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:28.481402 28592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "7bb6cbacf5df4e3780ce81917f92384e"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "9d4eae7727820f2dae7e03765cd84efa"
server_key_iv: "9e9aad42fae9ca8c881c322f19dca2a2"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.481793 28592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "7bb6cbacf5df4e3780ce81917f92384e"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "9d4eae7727820f2dae7e03765cd84efa"
server_key_iv: "9e9aad42fae9ca8c881c322f19dca2a2"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.483297 28592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:04:28.484150 28606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.484387 28592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:28.484485 28592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "7bb6cbacf5df4e3780ce81917f92384e"
format_stamp: "Formatted at 2026-09-25 22:04:28 on dist-test-slave-794d"
server_key: "9d4eae7727820f2dae7e03765cd84efa"
server_key_iv: "9e9aad42fae9ca8c881c322f19dca2a2"
server_key_version: "encryptionkey@0"
I20260925 22:04:28.484560 28592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:28.507951 28592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:28.508368 28592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:28.508526 28592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:28.508788 28592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:28.509207 28592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:28.509243 28592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.509275 28592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:28.509299 28592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:28.515884 28592 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:42327
I20260925 22:04:28.515951 28719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:42327 every 8 connection(s)
I20260925 22:04:28.516364 28592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:04:28.518837 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28592
I20260925 22:04:28.518962 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:04:28.519143 9665 external_mini_cluster.cc:1428] Setting key b764845d0da825078454295c76f264d0
I20260925 22:04:28.522197 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.517161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:46641 (local address 127.9.112.126:40525)
0925 22:04:28.517220 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.517222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.517486 (+ 264us) server_negotiation.cc:407] Connection header received
0925 22:04:28.518067 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.518068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.518082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.518138 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:28.518894 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.519565 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.520718 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.520936 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.520991 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.521879 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:28.521881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:28.521881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:28.521886 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:28.521906 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:28.521910 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:28.521953 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.522066 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.522105 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:04:28.522420 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.517157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40525 (local address 127.9.112.67:46641)
0925 22:04:28.517360 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.517398 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.517968 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.518218 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.518222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.518231 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:28.518811 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.518816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.519647 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.519648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.520636 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.520639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.521004 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.521005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.521146 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.521745 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:28.521746 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:28.521779 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:28.521805 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:28.521959 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:28.521965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:28.521968 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.522066 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.522182 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260925 22:04:28.522944 28720 heartbeater.cc:347] Connected to a master server at 127.9.112.126:40525
I20260925 22:04:28.523044 28720 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:28.523242 28720 heartbeater.cc:511] Master 127.9.112.126:40525 requested a full tablet report, sending...
I20260925 22:04:28.523671 28270 ts_manager.cc:194] Registered new tserver with Master: 7bb6cbacf5df4e3780ce81917f92384e (127.9.112.67:42327)
I20260925 22:04:28.524053 28270 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.67:46641
I20260925 22:04:28.531360 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:04:28.538110 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.533462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.9.112.126:40525)
0925 22:04:28.533538 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.533541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.533607 (+ 66us) server_negotiation.cc:407] Connection header received
0925 22:04:28.533688 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.533689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.533705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.533765 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:28.533935 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.534779 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.535797 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.536059 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.536115 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.536996 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:28.536998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:28.536999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:28.537004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:28.537023 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:28.537029 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:28.537150 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.537911 (+ 761us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.537925 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:04:28.540565 28270 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:04:28.543397 28270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:04:28.550369 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.546102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.9.112.65:44905)
0925 22:04:28.546760 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.546765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.546774 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:04:28.546815 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.546817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.546833 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.546909 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:28.547130 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.548253 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.549426 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.549953 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.550022 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.550133 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.550220 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.550264 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":98,"threads_started":1}
I20260925 22:04:28.550524 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.545894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:42327 (local address 127.0.0.1:45704)
0925 22:04:28.546111 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.546159 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.546242 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.546628 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.546632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.546634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:28.546735 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.546737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.548182 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.548183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.549134 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.549137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.550182 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.550183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.550261 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.550264 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.550303 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.550347 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260925 22:04:28.550637 28733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.546151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.9.112.66:40435)
0925 22:04:28.546378 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.546382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.546599 (+ 217us) server_negotiation.cc:407] Connection header received
0925 22:04:28.546713 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.546716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.546731 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.546787 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:28.546990 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.547758 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.549196 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.549989 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.550058 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.550146 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.550394 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.550547 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
I20260925 22:04:28.550805 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.546151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.9.112.67:42327)
0925 22:04:28.546336 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.546341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.546353 (+ 12us) server_negotiation.cc:407] Connection header received
0925 22:04:28.546465 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.546469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.546497 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.546590 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:28.546823 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.548082 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.549219 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.549991 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.550058 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.550146 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.550663 (+ 517us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.550716 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
I20260925 22:04:28.551015 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.545904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:40435 (local address 127.0.0.1:44058)
0925 22:04:28.546538 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.546546 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.546625 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.546885 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.546886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.546888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:28.546927 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.546928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.547839 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.547840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.549103 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.549106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.550184 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.550185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.550251 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.550257 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.550303 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.550347 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":7,"thread_start_us":67,"threads_started":1}
I20260925 22:04:28.551678 28654 tablet_service.cc:1467] Processing CreateTablet for tablet 77886d4ad4f04f0891301240d8d430e7 (DEFAULT_TABLE table=test-table [id=1e39855f0ec64925bc8f587cad28033e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:28.551674 28523 tablet_service.cc:1467] Processing CreateTablet for tablet 77886d4ad4f04f0891301240d8d430e7 (DEFAULT_TABLE table=test-table [id=1e39855f0ec64925bc8f587cad28033e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:28.552095 28523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77886d4ad4f04f0891301240d8d430e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:28.552116 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77886d4ad4f04f0891301240d8d430e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:28.555299 28734 tablet_bootstrap.cc:492] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Bootstrap starting.
I20260925 22:04:28.555513 28735 tablet_bootstrap.cc:492] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: Bootstrap starting.
I20260925 22:04:28.550355 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.545902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44905 (local address 127.0.0.1:54098)
0925 22:04:28.546429 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.546440 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.546485 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.546991 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.546992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.546995 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:28.547046 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.547048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.548363 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.548364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.549351 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.549353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.550036 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.550038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.550116 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.550130 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.550203 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.550263 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":116,"threads_started":1}
I20260925 22:04:28.556205 28734 tablet_bootstrap.cc:654] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:28.556213 28735 tablet_bootstrap.cc:654] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:28.556568 28734 log.cc:824] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:28.556579 28735 log.cc:824] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:28.556564 28392 tablet_service.cc:1467] Processing CreateTablet for tablet 77886d4ad4f04f0891301240d8d430e7 (DEFAULT_TABLE table=test-table [id=1e39855f0ec64925bc8f587cad28033e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:28.556990 28392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77886d4ad4f04f0891301240d8d430e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:28.557341 28735 tablet_bootstrap.cc:492] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: No bootstrap required, opened a new log
I20260925 22:04:28.557335 28734 tablet_bootstrap.cc:492] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: No bootstrap required, opened a new log
I20260925 22:04:28.557397 28734 ts_tablet_manager.cc:1397] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:04:28.557397 28735 ts_tablet_manager.cc:1397] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 22:04:28.559976 28738 tablet_bootstrap.cc:492] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: Bootstrap starting.
I20260925 22:04:28.560437 28735 raft_consensus.cc:348] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.560437 28734 raft_consensus.cc:348] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.560637 28735 raft_consensus.cc:374] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:28.560637 28734 raft_consensus.cc:374] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:28.560681 28735 raft_consensus.cc:729] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4fc08987d04bb5a44b08470966a504, State: Initialized, Role: FOLLOWER
I20260925 22:04:28.560681 28734 raft_consensus.cc:729] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb6cbacf5df4e3780ce81917f92384e, State: Initialized, Role: FOLLOWER
I20260925 22:04:28.560765 28738 tablet_bootstrap.cc:654] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:28.560854 28734 consensus_queue.cc:260] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.560854 28735 consensus_queue.cc:260] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.561111 28738 log.cc:824] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:28.561160 28734 ts_tablet_manager.cc:1428] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260925 22:04:28.561160 28735 ts_tablet_manager.cc:1428] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260925 22:04:28.561316 28589 heartbeater.cc:503] Master 127.9.112.126:40525 was elected leader, sending a full tablet report...
I20260925 22:04:28.561614 28720 heartbeater.cc:503] Master 127.9.112.126:40525 was elected leader, sending a full tablet report...
I20260925 22:04:28.562011 28738 tablet_bootstrap.cc:492] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: No bootstrap required, opened a new log
I20260925 22:04:28.562093 28738 ts_tablet_manager.cc:1397] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:28.564376 28738 raft_consensus.cc:348] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.564543 28738 raft_consensus.cc:374] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:28.564565 28738 raft_consensus.cc:729] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7089d76792ec4b68973881fc454387d1, State: Initialized, Role: FOLLOWER
I20260925 22:04:28.564677 28738 consensus_queue.cc:260] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.564946 28738 ts_tablet_manager.cc:1428] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:04:28.565047 28458 heartbeater.cc:503] Master 127.9.112.126:40525 was elected leader, sending a full tablet report...
W20260925 22:04:28.613343 28590 tablet.cc:2369] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:04:28.650336 28742 raft_consensus.cc:482] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:28.650547 28742 raft_consensus.cc:504] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.651016 28742 leader_election.cc:288] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb4fc08987d04bb5a44b08470966a504 (127.9.112.66:40435), 7bb6cbacf5df4e3780ce81917f92384e (127.9.112.67:42327)
I20260925 22:04:28.653970 28733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.651322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:55891 (local address 127.9.112.66:40435)
0925 22:04:28.651391 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.651393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.651407 (+ 14us) server_negotiation.cc:407] Connection header received
0925 22:04:28.651446 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.651447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.651463 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.651526 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:28.651694 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.652260 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.653251 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.653635 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.653701 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.653712 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.653843 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.653898 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:04:28.653970 28460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.651240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:40435 (local address 127.9.112.65:55891)
0925 22:04:28.651324 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.651337 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.651383 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.651533 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.651534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.651536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:28.651629 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.651631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.652357 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.652358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.653160 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.653162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.653703 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.653703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.653777 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.653782 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.653826 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.653878 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 22:04:28.654045 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.651303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:33723 (local address 127.9.112.67:42327)
0925 22:04:28.651359 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.651361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.651565 (+ 204us) server_negotiation.cc:407] Connection header received
0925 22:04:28.651607 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.651608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.651620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.651667 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:28.651808 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.652293 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.653309 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.653697 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.653763 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.653772 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.653925 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.653972 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:04:28.654078 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.651243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:42327 (local address 127.9.112.65:33723)
0925 22:04:28.651503 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.651510 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:28.651555 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:28.651673 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:28.651674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:28.651676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:28.651752 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.651754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.652402 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.652403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.653209 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:28.653211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.653771 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.653772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:28.653859 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.653862 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:28.653908 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:04:28.653948 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":26,"thread_start_us":176,"threads_started":1}
I20260925 22:04:28.654424 28543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77886d4ad4f04f0891301240d8d430e7" candidate_uuid: "7089d76792ec4b68973881fc454387d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4fc08987d04bb5a44b08470966a504" is_pre_election: true
I20260925 22:04:28.654486 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77886d4ad4f04f0891301240d8d430e7" candidate_uuid: "7089d76792ec4b68973881fc454387d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb6cbacf5df4e3780ce81917f92384e" is_pre_election: true
I20260925 22:04:28.654624 28543 raft_consensus.cc:2454] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7089d76792ec4b68973881fc454387d1 in term 0.
I20260925 22:04:28.654635 28674 raft_consensus.cc:2454] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7089d76792ec4b68973881fc454387d1 in term 0.
I20260925 22:04:28.654870 28345 leader_election.cc:302] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7089d76792ec4b68973881fc454387d1, 7bb6cbacf5df4e3780ce81917f92384e; no voters:
I20260925 22:04:28.655025 28742 raft_consensus.cc:2781] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:04:28.655085 28742 raft_consensus.cc:482] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:04:28.655109 28742 raft_consensus.cc:3037] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:28.655956 28742 raft_consensus.cc:504] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.656096 28742 leader_election.cc:288] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [CANDIDATE]: Term 1 election: Requested vote from peers bb4fc08987d04bb5a44b08470966a504 (127.9.112.66:40435), 7bb6cbacf5df4e3780ce81917f92384e (127.9.112.67:42327)
I20260925 22:04:28.656260 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77886d4ad4f04f0891301240d8d430e7" candidate_uuid: "7089d76792ec4b68973881fc454387d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb6cbacf5df4e3780ce81917f92384e"
I20260925 22:04:28.656312 28543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77886d4ad4f04f0891301240d8d430e7" candidate_uuid: "7089d76792ec4b68973881fc454387d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4fc08987d04bb5a44b08470966a504"
I20260925 22:04:28.656376 28674 raft_consensus.cc:3037] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:28.656418 28543 raft_consensus.cc:3037] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:28.657230 28674 raft_consensus.cc:2454] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7089d76792ec4b68973881fc454387d1 in term 1.
I20260925 22:04:28.657238 28543 raft_consensus.cc:2454] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7089d76792ec4b68973881fc454387d1 in term 1.
I20260925 22:04:28.657431 28345 leader_election.cc:302] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7089d76792ec4b68973881fc454387d1, 7bb6cbacf5df4e3780ce81917f92384e; no voters:
I20260925 22:04:28.657524 28742 raft_consensus.cc:2781] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:28.657709 28742 raft_consensus.cc:686] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 1 LEADER]: Becoming Leader. State: Replica: 7089d76792ec4b68973881fc454387d1, State: Running, Role: LEADER
I20260925 22:04:28.657817 28742 consensus_queue.cc:237] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } }
I20260925 22:04:28.658828 28268 catalog_manager.cc:5495] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7089d76792ec4b68973881fc454387d1 (127.9.112.65). New cstate: current_term: 1 leader_uuid: "7089d76792ec4b68973881fc454387d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7089d76792ec4b68973881fc454387d1" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 } health_report { overall_health: UNKNOWN } } }
I20260925 22:04:28.668324 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.665914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.9.112.65:44905)
0925 22:04:28.665978 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:28.665981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:28.665988 (+ 7us) server_negotiation.cc:407] Connection header received
0925 22:04:28.666062 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:28.666062 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:28.666085 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:28.666150 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:28.666288 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.666790 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.667534 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:28.667736 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:28.667792 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:28.667962 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:28.668079 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:28.668173 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:28.668209 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 22:04:28.668220 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:04:28.670928 28674 raft_consensus.cc:1261] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 1 FOLLOWER]: Refusing update from remote peer 7089d76792ec4b68973881fc454387d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:28.670925 28543 raft_consensus.cc:1261] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 1 FOLLOWER]: Refusing update from remote peer 7089d76792ec4b68973881fc454387d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:28.671401 28744 consensus_queue.cc:1035] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb6cbacf5df4e3780ce81917f92384e" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 42327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:28.671562 28742 consensus_queue.cc:1035] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4fc08987d04bb5a44b08470966a504" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 40435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:28.674996 28749 mvcc.cc:205] Tried to move back new op lower bound from 7333371366071791616 to 7333371366023233536. Current Snapshot: MvccSnapshot[applied={T|T < 7333371366071791616}]
I20260925 22:04:28.676971 28269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:28.677088 28269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:28.677939 28269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c5f0ceacae24bfc893f5ed995a12036: Sending DeleteTablet for 3 replicas of tablet 77886d4ad4f04f0891301240d8d430e7
I20260925 22:04:28.678331 28654 tablet_service.cc:1514] Processing DeleteTablet for tablet 77886d4ad4f04f0891301240d8d430e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:28 UTC) from {username='slave'} at 127.0.0.1:45704
I20260925 22:04:28.678354 28392 tablet_service.cc:1514] Processing DeleteTablet for tablet 77886d4ad4f04f0891301240d8d430e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:28 UTC) from {username='slave'} at 127.0.0.1:54098
I20260925 22:04:28.678383 28523 tablet_service.cc:1514] Processing DeleteTablet for tablet 77886d4ad4f04f0891301240d8d430e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:28 UTC) from {username='slave'} at 127.0.0.1:44058
I20260925 22:04:28.678614 28756 tablet_replica.cc:330] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1: stopping tablet replica
I20260925 22:04:28.678728 28756 raft_consensus.cc:2229] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:04:28.678867 28756 raft_consensus.cc:2258] T 77886d4ad4f04f0891301240d8d430e7 P 7089d76792ec4b68973881fc454387d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:28.678946 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28330
I20260925 22:04:28.678702 28758 tablet_replica.cc:330] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504: stopping tablet replica
I20260925 22:04:28.684708 28758 raft_consensus.cc:2229] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:28.684974 28758 raft_consensus.cc:2258] T 77886d4ad4f04f0891301240d8d430e7 P bb4fc08987d04bb5a44b08470966a504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:28.685256 28757 tablet_replica.cc:330] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: stopping tablet replica
I20260925 22:04:28.685402 28757 raft_consensus.cc:2229] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:28.685524 28757 raft_consensus.cc:2258] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 22:04:28.686043 28255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:44905 (error 108)
I20260925 22:04:28.686182 28757 ts_tablet_manager.cc:1905] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:28.686355 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28461
I20260925 22:04:28.691250 28757 ts_tablet_manager.cc:1918] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:28.691370 28757 log.cc:1196] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/77886d4ad4f04f0891301240d8d430e7
I20260925 22:04:28.691653 28757 ts_tablet_manager.cc:1939] T 77886d4ad4f04f0891301240d8d430e7 P 7bb6cbacf5df4e3780ce81917f92384e: Deleting consensus metadata
I20260925 22:04:28.696293 28254 catalog_manager.cc:4841] TS 7bb6cbacf5df4e3780ce81917f92384e (127.9.112.67:42327): tablet 77886d4ad4f04f0891301240d8d430e7 (table test-table [id=1e39855f0ec64925bc8f587cad28033e]) successfully deleted
I20260925 22:04:28.696836 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.696627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:44905 (local address 127.0.0.1:54124)
0925 22:04:28.696691 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.696784 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:44905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 22:04:28.697012 28255 catalog_manager.cc:4584] TS 7089d76792ec4b68973881fc454387d1 (127.9.112.65:44905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77886d4ad4f04f0891301240d8d430e7: Network error: Client connection negotiation failed: client connection to 127.9.112.65:44905: connect: Connection refused (error 111)
I20260925 22:04:28.700578 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28592
I20260925 22:04:28.700650 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:28.700514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:40435 (local address 127.0.0.1:44068)
0925 22:04:28.700571 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:28.700606 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.66:40435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 22:04:28.706200 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28239
2026-09-25T22:04:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (844 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:28 dist-test-slave-794d krb5kdc[28762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:28 dist-test-slave-794d krb5kdc[28762](info): set up 2 sockets
Sep 25 22:04:28 dist-test-slave-794d krb5kdc[28762](info): commencing operation
krb5kdc: starting...
W20260925 22:04:29.688681 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:29 dist-test-slave-794d krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:29.821774 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:42919
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:42919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:29.912007 28775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:29.912269 28775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:29.912304 28775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:29.913615 28775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:29.913666 28775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:29.913686 28775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:29.913700 28775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:29.913717 28775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:29.915356 28775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:42919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:42919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:29.915653 28775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:29.915892 28775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:29.921204 28780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:29.921200 28783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:29.921200 28781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:29.921489 28775 server_base.cc:1034] running on GCE node
I20260925 22:04:29.921722 28775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:29.922025 28775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:29.923173 28775 hybrid_clock.cc:648] HybridClock initialized: now 1790373869923156 us; error 30 us; skew 500 ppm
Sep 25 22:04:29 dist-test-slave-794d krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373869, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:29.926023 28775 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:29.926522 28775 webserver.cc:460] Webserver started at http://127.9.112.126:32803/ using document root <none> and password file <none>
I20260925 22:04:29.926760 28775 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:29.926810 28775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:29.926946 28775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:29.928243 28775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "711290d2644a4d16bc8a85a48e8dc7be"
format_stamp: "Formatted at 2026-09-25 22:04:29 on dist-test-slave-794d"
server_key: "2a512994c9cf95c1e30f2ee3045babb2"
server_key_iv: "be3f0c3319d48ff522133e2ae81951be"
server_key_version: "encryptionkey@0"
I20260925 22:04:29.928673 28775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "711290d2644a4d16bc8a85a48e8dc7be"
format_stamp: "Formatted at 2026-09-25 22:04:29 on dist-test-slave-794d"
server_key: "2a512994c9cf95c1e30f2ee3045babb2"
server_key_iv: "be3f0c3319d48ff522133e2ae81951be"
server_key_version: "encryptionkey@0"
I20260925 22:04:29.930124 28775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:29.930982 28790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:29.931166 28775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:29.931257 28775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "711290d2644a4d16bc8a85a48e8dc7be"
format_stamp: "Formatted at 2026-09-25 22:04:29 on dist-test-slave-794d"
server_key: "2a512994c9cf95c1e30f2ee3045babb2"
server_key_iv: "be3f0c3319d48ff522133e2ae81951be"
server_key_version: "encryptionkey@0"
I20260925 22:04:29.931339 28775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:29.964762 28775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:29.967784 28775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:29.968040 28775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:29.972795 28775 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:42919
I20260925 22:04:29.972888 28842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:42919 every 8 connection(s)
I20260925 22:04:29.973285 28775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:29.974327 28843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:29.977242 28843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be: Bootstrap starting.
I20260925 22:04:29.977977 28843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:29.978058 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28775
I20260925 22:04:29.978196 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:29.978312 28843 log.cc:824] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:29.978602 9665 external_mini_cluster.cc:1428] Setting key 007b03bee3e5bfebc92504c92e718198
I20260925 22:04:29.979195 28843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be: No bootstrap required, opened a new log
I20260925 22:04:29.981593 28843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711290d2644a4d16bc8a85a48e8dc7be" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42919 } }
I20260925 22:04:29.981793 28843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:29.981834 28843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711290d2644a4d16bc8a85a48e8dc7be, State: Initialized, Role: FOLLOWER
I20260925 22:04:29.981990 28843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711290d2644a4d16bc8a85a48e8dc7be" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42919 } }
I20260925 22:04:29.982086 28843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:29.982120 28843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:29.982165 28843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:29.983098 28843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711290d2644a4d16bc8a85a48e8dc7be" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42919 } }
I20260925 22:04:29.983260 28843 leader_election.cc:302] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 711290d2644a4d16bc8a85a48e8dc7be; no voters:
I20260925 22:04:29.983476 28843 leader_election.cc:288] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:29.983529 28848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:29.983685 28848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [term 1 LEADER]: Becoming Leader. State: Replica: 711290d2644a4d16bc8a85a48e8dc7be, State: Running, Role: LEADER
I20260925 22:04:29.983851 28848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711290d2644a4d16bc8a85a48e8dc7be" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42919 } }
I20260925 22:04:29.983882 28843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:29.984419 28849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 711290d2644a4d16bc8a85a48e8dc7be. Latest consensus state: current_term: 1 leader_uuid: "711290d2644a4d16bc8a85a48e8dc7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711290d2644a4d16bc8a85a48e8dc7be" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42919 } } }
I20260925 22:04:29.984508 28849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:29.984822 28850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "711290d2644a4d16bc8a85a48e8dc7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711290d2644a4d16bc8a85a48e8dc7be" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 42919 } } }
I20260925 22:04:29.984953 28850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:29.985471 28855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:29.986003 28855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 22:04:29 dist-test-slave-794d krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:29.987829 28855 catalog_manager.cc:1347] Generated new cluster ID: c0fe2d04778d4c4584a9f19ac206f5ee
I20260925 22:04:29.987871 28855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:29.993177 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:29.979302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.9.112.126:42919)
0925 22:04:29.979506 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:29.979517 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:29.979559 (+ 42us) server_negotiation.cc:407] Connection header received
0925 22:04:29.980720 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:29.980732 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:29.980826 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:29.981028 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:29.981721 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:29.983567 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:29.984776 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:29.985435 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:29.985571 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:29.988314 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:29.988319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:29.988319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:29.988332 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:29.989986 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:29.990329 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:29.990332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:29.990333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:29.990423 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:29.991214 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:29.991216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:29.991216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:29.991683 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:29.991893 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:29.992486 (+ 593us) server_negotiation.cc:299] Negotiation successful
0925 22:04:29.992809 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260925 22:04:29.996826 28855 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:29.997622 28855 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:30.005502 28855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be: Generated new TSK 0
I20260925 22:04:30.005762 28855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.112.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
Entry for principal kudu/127.9.112.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab.
I20260925 22:04:30.045804 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42919
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:30.152730 28869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:30.152987 28869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:30.153013 28869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:30.154282 28869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:30.154335 28869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:30.154363 28869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:30.155911 28869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.65.keytab
--principal=kudu/127.9.112.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:30.156179 28869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:30.156472 28869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:04:30.162112 28869 server_base.cc:1034] running on GCE node
W20260925 22:04:30.162073 28874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.162206 28877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.162081 28875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:30.162593 28869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:30.162869 28869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:30.164036 28869 hybrid_clock.cc:648] HybridClock initialized: now 1790373870164012 us; error 48 us; skew 500 ppm
Sep 25 22:04:30 dist-test-slave-794d krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373870, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:30.166977 28869 init.cc:377] Logged in from keytab as kudu/127.9.112.65@KRBTEST.COM (short username kudu)
I20260925 22:04:30.167500 28869 webserver.cc:460] Webserver started at http://127.9.112.65:41309/ using document root <none> and password file <none>
I20260925 22:04:30.167737 28869 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:30.167790 28869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:30.167940 28869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:30.169189 28869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "b7fc5ee567f0458f90939dcfb5bca530"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "96307ffd963bcefcce1306821bb5a887"
server_key_iv: "2386c71852f8af1414ce0d705d16dd23"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.169554 28869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "b7fc5ee567f0458f90939dcfb5bca530"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "96307ffd963bcefcce1306821bb5a887"
server_key_iv: "2386c71852f8af1414ce0d705d16dd23"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.170943 28869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:04:30.171770 28884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.171916 28869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 22:04:30.172029 28869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "b7fc5ee567f0458f90939dcfb5bca530"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "96307ffd963bcefcce1306821bb5a887"
server_key_iv: "2386c71852f8af1414ce0d705d16dd23"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.172097 28869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:30.189922 28869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:30.192410 28869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:30.192569 28869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:30.192827 28869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:30.193197 28869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:30.193239 28869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.193279 28869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:30.193298 28869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.199852 28869 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:43091
I20260925 22:04:30.199898 28997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:43091 every 8 connection(s)
I20260925 22:04:30.200290 28869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:30.203002 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 28869
I20260925 22:04:30.203135 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:04:30.203356 9665 external_mini_cluster.cc:1428] Setting key bc1a55d7bc11e4d6e4392ca8319f82ad
Sep 25 22:04:30 dist-test-slave-794d krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373870, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.65@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.66@KRBTEST.COM; defaulting to no policy
I20260925 22:04:30.213156 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.201275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:55351 (local address 127.9.112.126:42919)
0925 22:04:30.201384 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.201387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.201603 (+ 216us) server_negotiation.cc:407] Connection header received
0925 22:04:30.202273 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.202274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.202288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.202347 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:30.203682 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.204401 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.205555 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.205770 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.205832 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.208669 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:30.208672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:30.208672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:30.208681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:30.211085 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.211669 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.211670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.211671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.211737 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.212089 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.212091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.212091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.212239 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:30.212307 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.212792 (+ 485us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.213061 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 22:04:30.213249 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.201193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42919 (local address 127.9.112.65:55351)
0925 22:04:30.201449 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.201502 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.202156 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.202439 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.202442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.203242 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:30.203575 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.203581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.204533 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.204534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.205454 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.205457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.205853 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.205854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.205998 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.206687 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:30.206697 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:30.208564 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:30.211192 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:30.211204 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:30.211208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:30.211569 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:30.211815 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:30.211815 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:30.211816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:30.211988 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:30.212322 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:30.212330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:30.212591 (+ 261us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.212730 (+ 139us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.212949 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260925 22:04:30.214371 28998 heartbeater.cc:347] Connected to a master server at 127.9.112.126:42919
I20260925 22:04:30.214502 28998 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:30.214708 28998 heartbeater.cc:511] Master 127.9.112.126:42919 requested a full tablet report, sending...
I20260925 22:04:30.215451 28807 ts_manager.cc:194] Registered new tserver with Master: b7fc5ee567f0458f90939dcfb5bca530 (127.9.112.65:43091)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.66@KRBTEST.COM" created.
I20260925 22:04:30.216256 28807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.65@KRBTEST.COM'} at 127.9.112.65:55351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
Entry for principal kudu/127.9.112.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab.
I20260925 22:04:30.237241 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42919
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:30.333143 29003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:30.333459 29003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:30.333494 29003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:30.335150 29003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:30.335206 29003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:30.335237 29003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:30.338619 29003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.66.keytab
--principal=kudu/127.9.112.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:30.338986 29003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:30.339246 29003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:30.345117 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.345117 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.345117 29011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:30.346021 29003 server_base.cc:1034] running on GCE node
I20260925 22:04:30.346287 29003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:30.346587 29003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:30.347765 29003 hybrid_clock.cc:648] HybridClock initialized: now 1790373870347736 us; error 45 us; skew 500 ppm
Sep 25 22:04:30 dist-test-slave-794d krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373870, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:30.351737 29003 init.cc:377] Logged in from keytab as kudu/127.9.112.66@KRBTEST.COM (short username kudu)
I20260925 22:04:30.352470 29003 webserver.cc:460] Webserver started at http://127.9.112.66:35581/ using document root <none> and password file <none>
I20260925 22:04:30.352744 29003 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:30.352800 29003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:30.352943 29003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:30.354139 29003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "2cbac3f66a1081f3dc1d458cb5f1c0c1"
server_key_iv: "8b32c051b2476679a8c445f54c6a19e0"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.354573 29003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "2cbac3f66a1081f3dc1d458cb5f1c0c1"
server_key_iv: "8b32c051b2476679a8c445f54c6a19e0"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.356009 29003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:30.356976 29018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.357239 29003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 22:04:30.357352 29003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "2cbac3f66a1081f3dc1d458cb5f1c0c1"
server_key_iv: "8b32c051b2476679a8c445f54c6a19e0"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.357435 29003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:30.369709 29003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:30.372149 29003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:30.372304 29003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:30.372646 29003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:30.373018 29003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:30.373054 29003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.373088 29003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:30.373111 29003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.380635 29003 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:43959
I20260925 22:04:30.380688 29131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:43959 every 8 connection(s)
I20260925 22:04:30.381100 29003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:30.383934 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29003
I20260925 22:04:30.384056 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:04:30.384301 9665 external_mini_cluster.cc:1428] Setting key 0690e9dc403aabd9f6376fa69fdbeaeb
Sep 25 22:04:30 dist-test-slave-794d krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373870, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.66@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:30.391956 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.382060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:34677 (local address 127.9.112.126:42919)
0925 22:04:30.382213 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.382216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.382385 (+ 169us) server_negotiation.cc:407] Connection header received
0925 22:04:30.382997 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.382999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.383014 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.383068 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:30.384201 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.384912 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.386016 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.386231 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.386292 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.388956 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:30.388958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:30.388959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:30.388968 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:30.390545 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.390960 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.390961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.390962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.391023 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.391251 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.391252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.391252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.391391 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:30.391461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.391747 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.391847 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260925 22:04:30.392182 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.381959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42919 (local address 127.9.112.66:34677)
0925 22:04:30.382227 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.382278 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.382883 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.383148 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.383152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.383788 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:30.384087 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.384094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.385010 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.385011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.385921 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.385924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.386305 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.386306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.386457 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.387067 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:30.387078 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:30.388836 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:30.390631 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:30.390640 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:30.390643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:30.390883 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:30.391077 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:30.391078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:30.391078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:30.391198 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:30.391454 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:30.391461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:30.391624 (+ 163us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.391723 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.391916 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":100,"threads_started":1}
I20260925 22:04:30.392747 29132 heartbeater.cc:347] Connected to a master server at 127.9.112.126:42919
I20260925 22:04:30.392854 29132 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:30.393085 29132 heartbeater.cc:511] Master 127.9.112.126:42919 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.112.67@KRBTEST.COM; defaulting to no policy
I20260925 22:04:30.393520 28807 ts_manager.cc:194] Registered new tserver with Master: 3fc4d91ac6a14e4c8fd88be959cd40ea (127.9.112.66:43959)
I20260925 22:04:30.393940 28807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.66@KRBTEST.COM'} at 127.9.112.66:34677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
Entry for principal kudu/127.9.112.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab.
I20260925 22:04:30.416100 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42919
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:30.509405 29137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:30.509696 29137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:30.509734 29137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:30.511198 29137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:30.511273 29137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:30.511319 29137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:30.513242 29137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.67.keytab
--principal=kudu/127.9.112.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:30.513523 29137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:30.513767 29137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:30.519886 29145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:30.519867 29137 server_base.cc:1034] running on GCE node
W20260925 22:04:30.519837 29143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.519835 29142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:30.520370 29137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:30.520673 29137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:30.521807 29137 hybrid_clock.cc:648] HybridClock initialized: now 1790373870521775 us; error 53 us; skew 500 ppm
Sep 25 22:04:30 dist-test-slave-794d krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373870, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:30.524679 29137 init.cc:377] Logged in from keytab as kudu/127.9.112.67@KRBTEST.COM (short username kudu)
I20260925 22:04:30.525149 29137 webserver.cc:460] Webserver started at http://127.9.112.67:43513/ using document root <none> and password file <none>
I20260925 22:04:30.525378 29137 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:30.525426 29137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:30.525528 29137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:30.526687 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "8a809eeeac9c4055a6c52027aab313b3"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "5b2e9dc9bbfdb485aa4d41cdca25447d"
server_key_iv: "c828eb2d4e43765ca40df9c73c0480dc"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.527014 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "8a809eeeac9c4055a6c52027aab313b3"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "5b2e9dc9bbfdb485aa4d41cdca25447d"
server_key_iv: "c828eb2d4e43765ca40df9c73c0480dc"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.528537 29137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:04:30.529453 29152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.529652 29137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:30.529749 29137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "8a809eeeac9c4055a6c52027aab313b3"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "5b2e9dc9bbfdb485aa4d41cdca25447d"
server_key_iv: "c828eb2d4e43765ca40df9c73c0480dc"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.529817 29137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:30.544801 29137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:30.547400 29137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:30.547546 29137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:30.547784 29137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:30.548134 29137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:30.548170 29137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.548194 29137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:30.548205 29137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.555099 29137 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:43331
I20260925 22:04:30.555137 29265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:43331 every 8 connection(s)
I20260925 22:04:30.555516 29137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:04:30.562382 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29137
I20260925 22:04:30.562588 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:04:30.562839 9665 external_mini_cluster.cc:1428] Setting key 7104b7e391d79eaf80676be7e00f6e57
Sep 25 22:04:30 dist-test-slave-794d krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373870, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.67@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:30.566208 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.556409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:54913 (local address 127.9.112.126:42919)
0925 22:04:30.556507 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.556510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.556698 (+ 188us) server_negotiation.cc:407] Connection header received
0925 22:04:30.557299 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.557301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.557315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.557371 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:30.558574 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.559201 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.560310 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.560563 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.560624 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.563490 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:30.563492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:30.563492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:30.563501 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:30.564859 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.565256 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.565257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.565258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.565316 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.565544 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.565545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.565545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.565675 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:30.565732 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.566015 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.566106 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 22:04:30.566440 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.556330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:42919 (local address 127.9.112.67:54913)
0925 22:04:30.556563 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.556605 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.557182 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.557451 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.557455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.558091 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:30.558390 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.558396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.559294 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.559295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.560224 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.560228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.560637 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.560638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.560776 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.561406 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:30.561416 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:30.563382 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:30.564943 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:30.564951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:30.564953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:30.565189 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:30.565371 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:30.565371 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:30.565372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:30.565488 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:30.565731 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:30.565738 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:30.565892 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.565989 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.566180 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260925 22:04:30.567018 29266 heartbeater.cc:347] Connected to a master server at 127.9.112.126:42919
I20260925 22:04:30.567122 29266 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:30.567327 29266 heartbeater.cc:511] Master 127.9.112.126:42919 requested a full tablet report, sending...
I20260925 22:04:30.567785 28807 ts_manager.cc:194] Registered new tserver with Master: 8a809eeeac9c4055a6c52027aab313b3 (127.9.112.67:43331)
I20260925 22:04:30.568275 28807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.112.67@KRBTEST.COM'} at 127.9.112.67:54913
I20260925 22:04:30.575150 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:04:30.583910 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.576895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.9.112.126:42919)
0925 22:04:30.576970 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.576972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.577101 (+ 129us) server_negotiation.cc:407] Connection header received
0925 22:04:30.577149 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.577150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.577160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.577212 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:30.577809 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.578383 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.579283 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.579483 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.579562 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.580985 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:30.580987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:30.580987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:30.580996 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:30.582402 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.582864 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.582865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.582866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.582932 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:30.583218 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:30.583219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:30.583219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:30.583453 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:30.583573 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.583694 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.583783 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:04:30.585973 28807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:04:30.588501 28807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:04:30.596272 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.591896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.9.112.66:43959)
0925 22:04:30.592388 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.592392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.592403 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:30.592459 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.592462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.592480 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.592570 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:30.592763 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.593658 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.595008 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.595833 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.595897 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.595970 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.596127 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.596174 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":106,"threads_started":1}
I20260925 22:04:30.596266 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.591757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:43959 (local address 127.0.0.1:37694)
0925 22:04:30.591964 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.592019 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.592100 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.592589 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.592592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.592595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:30.592686 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.592689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.593752 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.593753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.594911 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.594913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.595909 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.595910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.596010 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.596027 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.596096 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.596164 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260925 22:04:30.596570 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.591759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:43091 (local address 127.0.0.1:57566)
0925 22:04:30.592524 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.592536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.592598 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.592927 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.592928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.592930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:30.592997 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.593000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.594385 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.594386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.595314 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.595316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.596242 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.596243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.596312 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.596317 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.596423 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.596478 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"spinlock_wait_cycles":768,"thread_start_us":131,"threads_started":1}
I20260925 22:04:30.596864 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.592152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.9.112.65:43091)
0925 22:04:30.592408 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.592415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.592655 (+ 240us) server_negotiation.cc:407] Connection header received
0925 22:04:30.592716 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.592719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.592735 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.592815 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:30.593132 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.594263 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.595390 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.596083 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.596152 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.596205 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.596705 (+ 500us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.596775 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260925 22:04:30.596900 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.591980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:43331 (local address 127.0.0.1:58414)
0925 22:04:30.592774 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.592783 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.592839 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.593023 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.593024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.593025 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:30.593069 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.593070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.594498 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.594499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.595576 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.595580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.596602 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.596603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.596751 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.596755 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.596798 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.596841 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":69,"threads_started":1}
I20260925 22:04:30.596969 29281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.592113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58414 (local address 127.9.112.67:43331)
0925 22:04:30.592300 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.592307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.592856 (+ 549us) server_negotiation.cc:407] Connection header received
0925 22:04:30.592912 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.592915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.592931 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.593018 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:30.593120 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.594353 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.595672 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.596457 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.596514 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.596568 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.596824 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.596877 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260925 22:04:30.597787 29200 tablet_service.cc:1467] Processing CreateTablet for tablet c2e801c79a864e4f98471fa8c441d9c8 (DEFAULT_TABLE table=test-table [id=cac41b97765b4f54a31813bb284621eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:30.597786 28932 tablet_service.cc:1467] Processing CreateTablet for tablet c2e801c79a864e4f98471fa8c441d9c8 (DEFAULT_TABLE table=test-table [id=cac41b97765b4f54a31813bb284621eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:30.597934 29066 tablet_service.cc:1467] Processing CreateTablet for tablet c2e801c79a864e4f98471fa8c441d9c8 (DEFAULT_TABLE table=test-table [id=cac41b97765b4f54a31813bb284621eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:30.598290 28932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e801c79a864e4f98471fa8c441d9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:30.598290 29200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e801c79a864e4f98471fa8c441d9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:30.598358 29066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e801c79a864e4f98471fa8c441d9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:30.601670 29283 tablet_bootstrap.cc:492] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Bootstrap starting.
I20260925 22:04:30.602380 29285 tablet_bootstrap.cc:492] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: Bootstrap starting.
I20260925 22:04:30.602517 29283 tablet_bootstrap.cc:654] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:30.603001 29285 tablet_bootstrap.cc:654] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:30.603052 29283 log.cc:824] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:30.603502 29285 log.cc:824] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:30.603811 29284 tablet_bootstrap.cc:492] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Bootstrap starting.
I20260925 22:04:30.603828 29283 tablet_bootstrap.cc:492] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: No bootstrap required, opened a new log
I20260925 22:04:30.603900 29283 ts_tablet_manager.cc:1397] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:30.604460 29284 tablet_bootstrap.cc:654] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:30.604758 29285 tablet_bootstrap.cc:492] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: No bootstrap required, opened a new log
I20260925 22:04:30.604838 29285 ts_tablet_manager.cc:1397] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 22:04:30.605079 29284 log.cc:824] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:30.605808 29283 raft_consensus.cc:348] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.605979 29283 raft_consensus.cc:374] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:30.606012 29283 raft_consensus.cc:729] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc4d91ac6a14e4c8fd88be959cd40ea, State: Initialized, Role: FOLLOWER
I20260925 22:04:30.606148 29283 consensus_queue.cc:260] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.606451 29283 ts_tablet_manager.cc:1428] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:04:30.606451 29132 heartbeater.cc:503] Master 127.9.112.126:42919 was elected leader, sending a full tablet report...
I20260925 22:04:30.606552 29285 raft_consensus.cc:348] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.606721 29285 raft_consensus.cc:374] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:30.606751 29285 raft_consensus.cc:729] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc5ee567f0458f90939dcfb5bca530, State: Initialized, Role: FOLLOWER
I20260925 22:04:30.606884 29285 consensus_queue.cc:260] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.607096 29284 tablet_bootstrap.cc:492] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: No bootstrap required, opened a new log
I20260925 22:04:30.607172 29285 ts_tablet_manager.cc:1428] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 22:04:30.607192 29284 ts_tablet_manager.cc:1397] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 22:04:30.607197 28998 heartbeater.cc:503] Master 127.9.112.126:42919 was elected leader, sending a full tablet report...
I20260925 22:04:30.609010 29284 raft_consensus.cc:348] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.609153 29284 raft_consensus.cc:374] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:30.609184 29284 raft_consensus.cc:729] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a809eeeac9c4055a6c52027aab313b3, State: Initialized, Role: FOLLOWER
I20260925 22:04:30.609292 29284 consensus_queue.cc:260] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.609581 29266 heartbeater.cc:503] Master 127.9.112.126:42919 was elected leader, sending a full tablet report...
I20260925 22:04:30.609653 29284 ts_tablet_manager.cc:1428] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260925 22:04:30.631845 29133 tablet.cc:2369] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:04:30.663666 29289 raft_consensus.cc:482] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:30.663874 29289 raft_consensus.cc:504] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.664417 29289 leader_election.cc:288] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a809eeeac9c4055a6c52027aab313b3 (127.9.112.67:43331), b7fc5ee567f0458f90939dcfb5bca530 (127.9.112.65:43091)
I20260925 22:04:30.667541 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.664858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:48015 (local address 127.9.112.65:43091)
0925 22:04:30.664911 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.664913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.665046 (+ 133us) server_negotiation.cc:407] Connection header received
0925 22:04:30.665084 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.665085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.665097 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.665147 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:30.665288 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.665790 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.666777 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.667192 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.667258 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.667269 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.667405 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.667451 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:04:30.667549 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.664709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:43091 (local address 127.9.112.66:48015)
0925 22:04:30.664971 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.664979 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.665032 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.665153 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.665153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.665155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:30.665229 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.665230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.665872 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.665873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.666680 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.666683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.667263 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.667264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.667340 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.667345 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.667389 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.667434 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":57,"thread_start_us":145,"threads_started":1}
I20260925 22:04:30.667708 29281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.664800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:60583 (local address 127.9.112.67:43331)
0925 22:04:30.664867 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.664870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.664883 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:04:30.664925 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.664926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.664937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.664996 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:30.665179 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.665704 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.666813 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.667229 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.667296 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.667305 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.667582 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.667637 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:04:30.667932 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.664709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:43331 (local address 127.9.112.66:60583)
0925 22:04:30.664763 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.664779 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:30.664853 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:30.665020 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:30.665021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:30.665024 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:30.665107 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.665110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.665789 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.665790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.666706 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:30.666708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.667303 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.667304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:30.667381 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.667383 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 22:04:30.667479 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 22:04:30.667530 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 22:04:30.668138 28952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e801c79a864e4f98471fa8c441d9c8" candidate_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc5ee567f0458f90939dcfb5bca530" is_pre_election: true
I20260925 22:04:30.668365 28952 raft_consensus.cc:2454] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc4d91ac6a14e4c8fd88be959cd40ea in term 0.
I20260925 22:04:30.668295 29220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e801c79a864e4f98471fa8c441d9c8" candidate_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a809eeeac9c4055a6c52027aab313b3" is_pre_election: true
I20260925 22:04:30.668541 29220 raft_consensus.cc:2454] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc4d91ac6a14e4c8fd88be959cd40ea in term 0.
I20260925 22:04:30.668607 29019 leader_election.cc:302] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc4d91ac6a14e4c8fd88be959cd40ea, b7fc5ee567f0458f90939dcfb5bca530; no voters:
I20260925 22:04:30.668761 29289 raft_consensus.cc:2781] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:04:30.668876 29289 raft_consensus.cc:482] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:04:30.668913 29289 raft_consensus.cc:3037] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:30.669775 29289 raft_consensus.cc:504] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.669919 29289 leader_election.cc:288] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [CANDIDATE]: Term 1 election: Requested vote from peers 8a809eeeac9c4055a6c52027aab313b3 (127.9.112.67:43331), b7fc5ee567f0458f90939dcfb5bca530 (127.9.112.65:43091)
I20260925 22:04:30.670089 29220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e801c79a864e4f98471fa8c441d9c8" candidate_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a809eeeac9c4055a6c52027aab313b3"
I20260925 22:04:30.670176 29220 raft_consensus.cc:3037] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:30.670171 28952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e801c79a864e4f98471fa8c441d9c8" candidate_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc5ee567f0458f90939dcfb5bca530"
I20260925 22:04:30.670272 28952 raft_consensus.cc:3037] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:30.671046 29220 raft_consensus.cc:2454] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc4d91ac6a14e4c8fd88be959cd40ea in term 1.
I20260925 22:04:30.671136 28952 raft_consensus.cc:2454] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc4d91ac6a14e4c8fd88be959cd40ea in term 1.
I20260925 22:04:30.671244 29022 leader_election.cc:302] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc4d91ac6a14e4c8fd88be959cd40ea, 8a809eeeac9c4055a6c52027aab313b3; no voters:
I20260925 22:04:30.671339 29289 raft_consensus.cc:2781] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:30.671533 29289 raft_consensus.cc:686] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 1 LEADER]: Becoming Leader. State: Replica: 3fc4d91ac6a14e4c8fd88be959cd40ea, State: Running, Role: LEADER
I20260925 22:04:30.671641 29289 consensus_queue.cc:237] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } }
I20260925 22:04:30.672711 28807 catalog_manager.cc:5495] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc4d91ac6a14e4c8fd88be959cd40ea (127.9.112.66). New cstate: current_term: 1 leader_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fc4d91ac6a14e4c8fd88be959cd40ea" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 43959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 } health_report { overall_health: UNKNOWN } } }
I20260925 22:04:30.692200 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.689474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.9.112.66:43959)
0925 22:04:30.689562 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.689564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.689573 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:04:30.689617 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.689618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.689640 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.689712 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:30.689852 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.690403 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.691230 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.691469 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.691532 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.691650 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:30.691779 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:30.691844 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.692037 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.692070 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 22:04:30.694516 28952 raft_consensus.cc:1261] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 1 FOLLOWER]: Refusing update from remote peer 3fc4d91ac6a14e4c8fd88be959cd40ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:30.694506 29220 raft_consensus.cc:1261] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 1 FOLLOWER]: Refusing update from remote peer 3fc4d91ac6a14e4c8fd88be959cd40ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:30.694918 29289 consensus_queue.cc:1035] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc5ee567f0458f90939dcfb5bca530" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 43091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:30.695107 29293 consensus_queue.cc:1035] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a809eeeac9c4055a6c52027aab313b3" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 43331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:30.698032 29296 mvcc.cc:205] Tried to move back new op lower bound from 7333371374361448448 to 7333371374271852544. Current Snapshot: MvccSnapshot[applied={T|T < 7333371374361448448}]
I20260925 22:04:30.698263 29298 mvcc.cc:205] Tried to move back new op lower bound from 7333371374361448448 to 7333371374271852544. Current Snapshot: MvccSnapshot[applied={T|T < 7333371374361448448}]
I20260925 22:04:30.699239 29297 mvcc.cc:205] Tried to move back new op lower bound from 7333371374361448448 to 7333371374271852544. Current Snapshot: MvccSnapshot[applied={T|T < 7333371374361448448}]
I20260925 22:04:30.701037 28807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:30.701148 28807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:30.702148 28807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 711290d2644a4d16bc8a85a48e8dc7be: Sending DeleteTablet for 3 replicas of tablet c2e801c79a864e4f98471fa8c441d9c8
I20260925 22:04:30.702654 29066 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e801c79a864e4f98471fa8c441d9c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:30 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:37694
I20260925 22:04:30.702667 29200 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e801c79a864e4f98471fa8c441d9c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:30 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:58414
I20260925 22:04:30.702940 29306 tablet_replica.cc:330] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: stopping tablet replica
I20260925 22:04:30.703011 28932 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e801c79a864e4f98471fa8c441d9c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:30 UTC) from {username='kudu', principal='kudu/127.9.112.126@KRBTEST.COM'} at 127.0.0.1:57566
I20260925 22:04:30.703085 29306 raft_consensus.cc:2229] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:30.703256 29306 raft_consensus.cc:2258] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:30.703244 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28869
I20260925 22:04:30.703258 29308 tablet_replica.cc:330] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530: stopping tablet replica
I20260925 22:04:30.703356 29308 raft_consensus.cc:2229] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:30.703467 29308 raft_consensus.cc:2258] T c2e801c79a864e4f98471fa8c441d9c8 P b7fc5ee567f0458f90939dcfb5bca530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:30.704617 29306 ts_tablet_manager.cc:1905] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:30.706785 29307 tablet_replica.cc:330] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: stopping tablet replica
I20260925 22:04:30.706998 29307 raft_consensus.cc:2229] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:04:30.707005 29306 ts_tablet_manager.cc:1918] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:30.707100 29306 log.cc:1196] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/c2e801c79a864e4f98471fa8c441d9c8
I20260925 22:04:30.707283 29307 raft_consensus.cc:2258] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:30.707607 29306 ts_tablet_manager.cc:1939] T c2e801c79a864e4f98471fa8c441d9c8 P 8a809eeeac9c4055a6c52027aab313b3: Deleting consensus metadata
I20260925 22:04:30.707840 29307 ts_tablet_manager.cc:1905] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:30.708158 28794 catalog_manager.cc:4841] TS 8a809eeeac9c4055a6c52027aab313b3 (127.9.112.67:43331): tablet c2e801c79a864e4f98471fa8c441d9c8 (table test-table [id=cac41b97765b4f54a31813bb284621eb]) successfully deleted
I20260925 22:04:30.709923 29307 ts_tablet_manager.cc:1918] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:30.710064 29307 log.cc:1196] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/c2e801c79a864e4f98471fa8c441d9c8
I20260925 22:04:30.710397 29307 ts_tablet_manager.cc:1939] T c2e801c79a864e4f98471fa8c441d9c8 P 3fc4d91ac6a14e4c8fd88be959cd40ea: Deleting consensus metadata
I20260925 22:04:30.712589 28794 catalog_manager.cc:4841] TS 3fc4d91ac6a14e4c8fd88be959cd40ea (127.9.112.66:43959): tablet c2e801c79a864e4f98471fa8c441d9c8 (table test-table [id=cac41b97765b4f54a31813bb284621eb]) successfully deleted
W20260925 22:04:30.712805 28791 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:43091 (error 108)
I20260925 22:04:30.713426 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.713287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:43091 (local address 127.0.0.1:57582)
0925 22:04:30.713344 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:30.713387 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 22:04:30.713554 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29003
W20260925 22:04:30.714576 28791 catalog_manager.cc:4584] TS b7fc5ee567f0458f90939dcfb5bca530 (127.9.112.65:43091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2e801c79a864e4f98471fa8c441d9c8: Network error: Client connection negotiation failed: client connection to 127.9.112.65:43091: connect: Connection refused (error 111)
I20260925 22:04:30.719517 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29137
I20260925 22:04:30.724556 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 28775
2026-09-25T22:04:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2017 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T22:04:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:30Z Disabled control of system clock
I20260925 22:04:30.753784 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35715
--webserver_interface=127.9.112.126
--webserver_port=0
--builtin_ntp_servers=127.9.112.84:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:30.850405 29313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:30.850669 29313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:30.850690 29313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:30.851908 29313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:30.851950 29313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:30.851961 29313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:30.851972 29313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:30.851984 29313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:30.853426 29313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:30.853704 29313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:30.854031 29313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:30.859315 29321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.859380 29318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:30.859315 29319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:30.859511 29313 server_base.cc:1034] running on GCE node
I20260925 22:04:30.859848 29313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:30.860180 29313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:30.861380 29313 hybrid_clock.cc:648] HybridClock initialized: now 1790373870861361 us; error 33 us; skew 500 ppm
I20260925 22:04:30.862808 29313 webserver.cc:460] Webserver started at http://127.9.112.126:34189/ using document root <none> and password file <none>
I20260925 22:04:30.863118 29313 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:30.863181 29313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:30.863322 29313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:30.864585 29313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "71e00014fd6143e4a6e9bec4496b2e30"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "9f2fd80efab44ea5ed8874bc2fa2c847"
server_key_iv: "3ead16bbf6d62ab74b38c99fbc7b1ef1"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.864965 29313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "71e00014fd6143e4a6e9bec4496b2e30"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "9f2fd80efab44ea5ed8874bc2fa2c847"
server_key_iv: "3ead16bbf6d62ab74b38c99fbc7b1ef1"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.866303 29313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:30.867121 29327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:30.867295 29313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:30.867375 29313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "71e00014fd6143e4a6e9bec4496b2e30"
format_stamp: "Formatted at 2026-09-25 22:04:30 on dist-test-slave-794d"
server_key: "9f2fd80efab44ea5ed8874bc2fa2c847"
server_key_iv: "3ead16bbf6d62ab74b38c99fbc7b1ef1"
server_key_version: "encryptionkey@0"
I20260925 22:04:30.867443 29313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:30.890204 29313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:30.890633 29313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:30.890789 29313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:30.895381 29313 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35715
I20260925 22:04:30.895454 29379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35715 every 8 connection(s)
I20260925 22:04:30.895824 29313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:30.897002 29380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:30.900020 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29313
I20260925 22:04:30.900139 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:30.900207 29380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30: Bootstrap starting.
I20260925 22:04:30.900398 9665 external_mini_cluster.cc:1428] Setting key b505f224d09e648fc7a25e960588e26d
I20260925 22:04:30.901031 29380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:30.901437 29380 log.cc:824] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:30.902231 29380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30: No bootstrap required, opened a new log
I20260925 22:04:30.904896 29380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e00014fd6143e4a6e9bec4496b2e30" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35715 } }
I20260925 22:04:30.905102 29380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:30.905149 29380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e00014fd6143e4a6e9bec4496b2e30, State: Initialized, Role: FOLLOWER
I20260925 22:04:30.905298 29380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e00014fd6143e4a6e9bec4496b2e30" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35715 } }
I20260925 22:04:30.905376 29380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:30.905412 29380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:30.905460 29380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:30.906345 29380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e00014fd6143e4a6e9bec4496b2e30" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35715 } }
I20260925 22:04:30.906456 29380 leader_election.cc:302] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71e00014fd6143e4a6e9bec4496b2e30; no voters:
I20260925 22:04:30.906615 29380 leader_election.cc:288] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:30.906741 29385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:30.906942 29385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [term 1 LEADER]: Becoming Leader. State: Replica: 71e00014fd6143e4a6e9bec4496b2e30, State: Running, Role: LEADER
I20260925 22:04:30.907115 29380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:30.907121 29385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e00014fd6143e4a6e9bec4496b2e30" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35715 } }
I20260925 22:04:30.907573 29385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71e00014fd6143e4a6e9bec4496b2e30. Latest consensus state: current_term: 1 leader_uuid: "71e00014fd6143e4a6e9bec4496b2e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e00014fd6143e4a6e9bec4496b2e30" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35715 } } }
I20260925 22:04:30.907617 29387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71e00014fd6143e4a6e9bec4496b2e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e00014fd6143e4a6e9bec4496b2e30" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35715 } } }
I20260925 22:04:30.907650 29385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:30.907675 29387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:30.907938 29392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:30.908162 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:30.901021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.9.112.126:35715)
0925 22:04:30.901263 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:30.901273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:30.901312 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:04:30.901890 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:30.901900 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:30.901997 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:30.902140 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:30.902389 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.903949 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.905043 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:30.905709 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:30.905840 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:30.906846 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:30.906850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:30.906850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:30.907184 (+ 334us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:30.907236 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:30.907254 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:30.907639 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:30.907679 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 22:04:30.907792 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260925 22:04:30.908578 29392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:30.910531 29392 catalog_manager.cc:1347] Generated new cluster ID: 25bfc826b9864efcaa320207822b19f7
I20260925 22:04:30.910584 29392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:30.917500 29392 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:30.918141 29392 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:30.937016 29392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30: Generated new TSK 0
I20260925 22:04:30.937386 29392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:30.940404 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.65:0
--local_ip_for_outbound_sockets=127.9.112.65
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35715
--builtin_ntp_servers=127.9.112.84:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:31.036320 29404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:31.036638 29404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:31.036674 29404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:31.038296 29404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:31.038376 29404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:31.038420 29404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.65
I20260925 22:04:31.040241 29404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.112.65
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.65
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:31.040618 29404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:31.041026 29404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:31.046620 29409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:31.046622 29410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:31.046756 29412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:31.047174 29404 server_base.cc:1034] running on GCE node
I20260925 22:04:31.047374 29404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:31.047636 29404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:31.048779 29404 hybrid_clock.cc:648] HybridClock initialized: now 1790373871048767 us; error 31 us; skew 500 ppm
I20260925 22:04:31.050189 29404 webserver.cc:460] Webserver started at http://127.9.112.65:39611/ using document root <none> and password file <none>
I20260925 22:04:31.050480 29404 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:31.050541 29404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:31.050680 29404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:31.051923 29404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data/instance:
uuid: "ee622e41361d49dc8209224d2a3acde5"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "1a09ca831ab7389fcc8f4619a1e8a757"
server_key_iv: "04aeb5d40df6871c49410139455ebc9a"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.052297 29404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance:
uuid: "ee622e41361d49dc8209224d2a3acde5"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "1a09ca831ab7389fcc8f4619a1e8a757"
server_key_iv: "04aeb5d40df6871c49410139455ebc9a"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.053710 29404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:31.054625 29418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.054800 29404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:31.054893 29404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal
uuid: "ee622e41361d49dc8209224d2a3acde5"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "1a09ca831ab7389fcc8f4619a1e8a757"
server_key_iv: "04aeb5d40df6871c49410139455ebc9a"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.054976 29404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:31.062144 29404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:31.062600 29404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:31.062757 29404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:31.063006 29404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:31.063408 29404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:31.063445 29404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.063478 29404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:31.063503 29404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.070411 29404 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.65:37635
I20260925 22:04:31.070472 29531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.65:37635 every 8 connection(s)
I20260925 22:04:31.070878 29404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/data/info.pb
I20260925 22:04:31.076743 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29404
I20260925 22:04:31.076849 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-0/wal/instance
I20260925 22:04:31.076890 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.071866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.65:50279 (local address 127.9.112.126:35715)
0925 22:04:31.071934 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.071937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.072242 (+ 305us) server_negotiation.cc:407] Connection header received
0925 22:04:31.072937 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.072938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.072953 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.073008 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:31.073753 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.074304 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.075340 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.075556 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.075615 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.076598 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:31.076600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:31.076601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:31.076608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:31.076628 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:31.076632 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:31.076677 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.076759 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.076793 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 22:04:31.077067 9665 external_mini_cluster.cc:1428] Setting key 3023e0a9309d12b5e6a56c338bc28d7d
I20260925 22:04:31.077193 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.071865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35715 (local address 127.9.112.65:50279)
0925 22:04:31.072091 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.072140 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.072819 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.073091 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.073096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.073105 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:31.073663 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.073668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.074393 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.074395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.075246 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.075249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.075636 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.075637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.075771 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.076421 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:31.076422 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:31.076464 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:31.076492 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:31.076685 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:31.076691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:31.076695 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.076769 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.076894 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260925 22:04:31.077786 29532 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35715
I20260925 22:04:31.077893 29532 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:31.078136 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.66:0
--local_ip_for_outbound_sockets=127.9.112.66
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35715
--builtin_ntp_servers=127.9.112.84:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 22:04:31.078168 29532 heartbeater.cc:511] Master 127.9.112.126:35715 requested a full tablet report, sending...
I20260925 22:04:31.078806 29344 ts_manager.cc:194] Registered new tserver with Master: ee622e41361d49dc8209224d2a3acde5 (127.9.112.65:37635)
I20260925 22:04:31.079679 29344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.65:50279
W20260925 22:04:31.174978 29535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:31.175235 29535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:31.175282 29535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:31.176641 29535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:31.176684 29535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:31.176719 29535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.66
I20260925 22:04:31.178238 29535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.112.66
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.66
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:31.178529 29535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:31.178879 29535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:31.184526 29540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:31.184625 29541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:31.184533 29543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:31.184834 29535 server_base.cc:1034] running on GCE node
I20260925 22:04:31.185102 29535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:31.185361 29535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:31.186529 29535 hybrid_clock.cc:648] HybridClock initialized: now 1790373871186501 us; error 45 us; skew 500 ppm
I20260925 22:04:31.187970 29535 webserver.cc:460] Webserver started at http://127.9.112.66:42633/ using document root <none> and password file <none>
I20260925 22:04:31.188220 29535 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:31.188267 29535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:31.188418 29535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:31.189608 29535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data/instance:
uuid: "85881b3516aa4f4490532cb25784f6c3"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "217673b90ae3577d1aef747d849ce1a3"
server_key_iv: "4529485c8d7ac961c83731a562faf9ed"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.189956 29535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance:
uuid: "85881b3516aa4f4490532cb25784f6c3"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "217673b90ae3577d1aef747d849ce1a3"
server_key_iv: "4529485c8d7ac961c83731a562faf9ed"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.191316 29535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 22:04:31.192145 29549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.192380 29535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:31.192481 29535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal
uuid: "85881b3516aa4f4490532cb25784f6c3"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "217673b90ae3577d1aef747d849ce1a3"
server_key_iv: "4529485c8d7ac961c83731a562faf9ed"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.192569 29535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:31.218348 29535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:31.218786 29535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:31.218954 29535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:31.219238 29535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:31.219624 29535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:31.219660 29535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.219694 29535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:31.219714 29535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.226282 29535 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.66:45981
I20260925 22:04:31.226348 29662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.66:45981 every 8 connection(s)
I20260925 22:04:31.226738 29535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/data/info.pb
I20260925 22:04:31.233129 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.227612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.66:53777 (local address 127.9.112.126:35715)
0925 22:04:31.227673 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.227676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.227918 (+ 242us) server_negotiation.cc:407] Connection header received
0925 22:04:31.228626 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.228628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.228644 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.228702 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:31.229459 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.230022 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.231448 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.231660 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.231725 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.232812 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:31.232814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:31.232815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:31.232821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:31.232840 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:31.232844 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:31.232890 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.232990 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.233027 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:04:31.233368 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.227580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35715 (local address 127.9.112.66:53777)
0925 22:04:31.227776 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.227819 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.228499 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.228780 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.228784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.228793 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:31.229363 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.229367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.230122 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.230122 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.231305 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.231309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.231743 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.231744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.231894 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.232622 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:31.232623 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:31.232658 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:31.232686 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:31.232907 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:31.232913 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:31.232916 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.232992 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.233114 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260925 22:04:31.234076 29663 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35715
I20260925 22:04:31.234189 29663 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:31.234503 29663 heartbeater.cc:511] Master 127.9.112.126:35715 requested a full tablet report, sending...
I20260925 22:04:31.235138 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29535
I20260925 22:04:31.235152 29344 ts_manager.cc:194] Registered new tserver with Master: 85881b3516aa4f4490532cb25784f6c3 (127.9.112.66:45981)
I20260925 22:04:31.235242 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal/instance
I20260925 22:04:31.235558 9665 external_mini_cluster.cc:1428] Setting key 0b5c599320c97d5730c55e57aeb6cb89
I20260925 22:04:31.235594 29344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.66:53777
I20260925 22:04:31.236636 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.112.67:0
--local_ip_for_outbound_sockets=127.9.112.67
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35715
--builtin_ntp_servers=127.9.112.84:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 22:04:31.329869 29666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:31.330111 29666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:31.330142 29666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:31.331379 29666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:31.331422 29666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:31.331455 29666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.112.67
I20260925 22:04:31.332954 29666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.112.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.112.67
--webserver_port=0
--tserver_master_addrs=127.9.112.126:35715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.112.67
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:31.333235 29666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:31.333572 29666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:31.339480 29674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:31.339480 29671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:31.339475 29672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:31.339545 29666 server_base.cc:1034] running on GCE node
I20260925 22:04:31.339900 29666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:31.340176 29666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:31.341342 29666 hybrid_clock.cc:648] HybridClock initialized: now 1790373871341338 us; error 71 us; skew 500 ppm
I20260925 22:04:31.342939 29666 webserver.cc:460] Webserver started at http://127.9.112.67:34641/ using document root <none> and password file <none>
I20260925 22:04:31.343190 29666 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:31.343240 29666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:31.343379 29666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:31.344626 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data/instance:
uuid: "b029046c90fe4b14bfdb8c98f6ba528c"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "c1225e9052ffaa8fff72563fda0606a4"
server_key_iv: "29aec915b39c04552a9cac85c1018d6d"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.344985 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance:
uuid: "b029046c90fe4b14bfdb8c98f6ba528c"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "c1225e9052ffaa8fff72563fda0606a4"
server_key_iv: "29aec915b39c04552a9cac85c1018d6d"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.346283 29666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:31.347030 29680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.347220 29666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:31.347306 29666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal
uuid: "b029046c90fe4b14bfdb8c98f6ba528c"
format_stamp: "Formatted at 2026-09-25 22:04:31 on dist-test-slave-794d"
server_key: "c1225e9052ffaa8fff72563fda0606a4"
server_key_iv: "29aec915b39c04552a9cac85c1018d6d"
server_key_version: "encryptionkey@0"
I20260925 22:04:31.347368 29666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:31.365979 29666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:31.366366 29666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:31.366513 29666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:31.366766 29666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 22:04:31.367368 29666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 22:04:31.367421 29666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.367456 29666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 22:04:31.367475 29666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:31.374296 29666 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.67:40631
I20260925 22:04:31.374351 29793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.67:40631 every 8 connection(s)
I20260925 22:04:31.374752 29666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/data/info.pb
I20260925 22:04:31.381032 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.375611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:40061 (local address 127.9.112.126:35715)
0925 22:04:31.375684 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.375687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.375992 (+ 305us) server_negotiation.cc:407] Connection header received
0925 22:04:31.376662 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.376664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.376681 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.376740 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:31.377548 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.378099 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.379159 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.379372 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.379434 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.380642 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:31.380644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:31.380645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:31.380652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:31.380673 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:31.380679 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:31.380727 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.380875 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.380931 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 22:04:31.381251 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.375577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:35715 (local address 127.9.112.67:40061)
0925 22:04:31.375781 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.375889 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.376549 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.376843 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.376847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.376857 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:31.377446 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.377450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.378196 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.378197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.379066 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.379069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.379455 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.379456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.379598 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.380429 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 22:04:31.380430 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:31.380475 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 22:04:31.380509 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:31.380744 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:31.380751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:31.380754 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.380875 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.380999 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260925 22:04:31.381716 29794 heartbeater.cc:347] Connected to a master server at 127.9.112.126:35715
I20260925 22:04:31.381822 29794 heartbeater.cc:464] Registering TS with master...
I20260925 22:04:31.382033 29794 heartbeater.cc:511] Master 127.9.112.126:35715 requested a full tablet report, sending...
I20260925 22:04:31.382471 29344 ts_manager.cc:194] Registered new tserver with Master: b029046c90fe4b14bfdb8c98f6ba528c (127.9.112.67:40631)
I20260925 22:04:31.382902 29344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.112.67:40061
I20260925 22:04:31.382860 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29666
I20260925 22:04:31.382994 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal/instance
I20260925 22:04:31.383263 9665 external_mini_cluster.cc:1428] Setting key eb0874ba78d580a5d5587c15f02c2c8e
I20260925 22:04:31.384217 9665 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 22:04:31.388923 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.385892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.9.112.126:35715)
0925 22:04:31.385975 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.385978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.386064 (+ 86us) server_negotiation.cc:407] Connection header received
0925 22:04:31.386142 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.386143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.386155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.386203 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:31.386359 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.386873 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.387648 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.387845 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.387901 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.388676 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:31.388678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:31.388679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:31.388684 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:31.388702 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:31.388707 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:31.388747 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.388805 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.388831 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 22:04:31.390550 29344 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 22:04:31.392549 29344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 22:04:31.399425 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.395095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45981 (local address 127.0.0.1:48228)
0925 22:04:31.395790 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.395800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.395861 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.396102 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.396103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.396105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:31.396160 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.396161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.397447 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.397448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.398272 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.398274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.399094 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.399095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.399172 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.399186 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.399267 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.399323 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260925 22:04:31.399422 29807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.395453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.9.112.66:45981)
0925 22:04:31.395739 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.395744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.395863 (+ 119us) server_negotiation.cc:407] Connection header received
0925 22:04:31.395937 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.395940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.395956 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.396029 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:31.396229 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.397378 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.398362 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.399015 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.399096 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.399199 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.399282 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.399323 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
I20260925 22:04:31.399789 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.395095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37635 (local address 127.0.0.1:33758)
0925 22:04:31.395932 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.395940 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.396018 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.396182 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.396183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.396185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:31.396245 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.396247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.397298 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.397299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.398596 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.398599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.399574 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.399575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.399635 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.399639 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.399684 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.399724 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":98,"thread_start_us":102,"threads_started":1}
I20260925 22:04:31.400009 29805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.395218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33758 (local address 127.9.112.65:37635)
0925 22:04:31.395462 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.395467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.396005 (+ 538us) server_negotiation.cc:407] Connection header received
0925 22:04:31.396090 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.396092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.396108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.396183 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:31.396298 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.397199 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.398697 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.399388 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.399429 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.399515 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.399704 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.399744 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260925 22:04:31.400413 29597 tablet_service.cc:1467] Processing CreateTablet for tablet 40ed54e005694580b361dda5aa91cc6a (DEFAULT_TABLE table=test-table [id=ed533feac73846279f36c082300be79f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:31.400693 29804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.395095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.67:40631 (local address 127.0.0.1:39256)
0925 22:04:31.395441 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.395487 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.395572 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.395758 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.395761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.395763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:31.395845 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.395847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.397277 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.397278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.398820 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.398822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.400458 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.400459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.400526 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.400530 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.400578 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.400625 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":34,"thread_start_us":97,"threads_started":1}
I20260925 22:04:31.400707 29466 tablet_service.cc:1467] Processing CreateTablet for tablet 40ed54e005694580b361dda5aa91cc6a (DEFAULT_TABLE table=test-table [id=ed533feac73846279f36c082300be79f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:31.400900 29597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40ed54e005694580b361dda5aa91cc6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:31.401036 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.395397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.9.112.67:40631)
0925 22:04:31.395581 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.395586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.395597 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:31.395659 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.395661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.395678 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.395749 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:31.395932 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.397190 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.398975 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.400238 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.400301 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.400422 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.400892 (+ 470us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.400947 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260925 22:04:31.401095 29466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40ed54e005694580b361dda5aa91cc6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:31.401682 29728 tablet_service.cc:1467] Processing CreateTablet for tablet 40ed54e005694580b361dda5aa91cc6a (DEFAULT_TABLE table=test-table [id=ed533feac73846279f36c082300be79f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 22:04:31.402045 29728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40ed54e005694580b361dda5aa91cc6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:31.404053 29808 tablet_bootstrap.cc:492] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Bootstrap starting.
I20260925 22:04:31.404831 29808 tablet_bootstrap.cc:654] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:31.405125 29808 log.cc:824] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:31.405444 29811 tablet_bootstrap.cc:492] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Bootstrap starting.
I20260925 22:04:31.405565 29809 tablet_bootstrap.cc:492] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: Bootstrap starting.
I20260925 22:04:31.405823 29808 tablet_bootstrap.cc:492] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: No bootstrap required, opened a new log
I20260925 22:04:31.405892 29808 ts_tablet_manager.cc:1397] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:04:31.406239 29809 tablet_bootstrap.cc:654] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:31.406263 29811 tablet_bootstrap.cc:654] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:31.406625 29811 log.cc:824] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:31.406623 29809 log.cc:824] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:31.407342 29809 tablet_bootstrap.cc:492] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: No bootstrap required, opened a new log
I20260925 22:04:31.407342 29811 tablet_bootstrap.cc:492] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: No bootstrap required, opened a new log
I20260925 22:04:31.407423 29809 ts_tablet_manager.cc:1397] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 22:04:31.407500 29811 ts_tablet_manager.cc:1397] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:31.408259 29808 raft_consensus.cc:348] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.408452 29808 raft_consensus.cc:374] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:31.408486 29808 raft_consensus.cc:729] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85881b3516aa4f4490532cb25784f6c3, State: Initialized, Role: FOLLOWER
I20260925 22:04:31.408589 29808 consensus_queue.cc:260] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.408854 29808 ts_tablet_manager.cc:1428] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 22:04:31.410063 29809 raft_consensus.cc:348] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.410238 29809 raft_consensus.cc:374] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:31.410269 29809 raft_consensus.cc:729] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee622e41361d49dc8209224d2a3acde5, State: Initialized, Role: FOLLOWER
I20260925 22:04:31.410392 29809 consensus_queue.cc:260] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.410439 29811 raft_consensus.cc:348] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.410585 29811 raft_consensus.cc:374] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:31.410625 29811 raft_consensus.cc:729] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b029046c90fe4b14bfdb8c98f6ba528c, State: Initialized, Role: FOLLOWER
I20260925 22:04:31.410687 29809 ts_tablet_manager.cc:1428] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 22:04:31.410732 29811 consensus_queue.cc:260] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.410763 29532 heartbeater.cc:503] Master 127.9.112.126:35715 was elected leader, sending a full tablet report...
I20260925 22:04:31.410905 29663 heartbeater.cc:503] Master 127.9.112.126:35715 was elected leader, sending a full tablet report...
I20260925 22:04:31.411041 29794 heartbeater.cc:503] Master 127.9.112.126:35715 was elected leader, sending a full tablet report...
I20260925 22:04:31.410974 29811 ts_tablet_manager.cc:1428] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260925 22:04:31.477429 29664 tablet.cc:2369] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:04:31.571862 29533 tablet.cc:2369] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 22:04:31.625473 29795 tablet.cc:2369] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 22:04:31.659286 29816 raft_consensus.cc:482] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 22:04:31.659476 29816 raft_consensus.cc:504] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.659991 29816 leader_election.cc:288] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85881b3516aa4f4490532cb25784f6c3 (127.9.112.66:45981), ee622e41361d49dc8209224d2a3acde5 (127.9.112.65:37635)
I20260925 22:04:31.664228 29805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.660299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:39729 (local address 127.9.112.65:37635)
0925 22:04:31.660392 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.660394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.660615 (+ 221us) server_negotiation.cc:407] Connection header received
0925 22:04:31.660650 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.660651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.660666 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.660714 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:31.660870 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.661582 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.663200 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.663794 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.663900 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.663914 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.664102 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.664153 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 22:04:31.664223 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.660228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37635 (local address 127.9.112.67:39729)
0925 22:04:31.660519 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.660536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.660606 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.660732 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.660733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.660734 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:31.660808 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.660809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.661685 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.661687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.663092 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.663096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.663938 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.663940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.664025 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.664030 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.664085 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.664125 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":104,"thread_start_us":195,"threads_started":1}
I20260925 22:04:31.664424 29807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.660328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.112.67:42225 (local address 127.9.112.66:45981)
0925 22:04:31.660418 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.660421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.660452 (+ 31us) server_negotiation.cc:407] Connection header received
0925 22:04:31.660496 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.660497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.660511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.660553 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 22:04:31.660768 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.661587 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.663201 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.663794 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.663916 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.663925 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.664280 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.664333 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 22:04:31.664235 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.660228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.66:45981 (local address 127.9.112.67:42225)
0925 22:04:31.660308 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.660322 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:31.660398 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:31.660586 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:31.660587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:31.660589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 22:04:31.660684 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.660687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.661685 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.661687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.663092 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:31.663096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.663968 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.663969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:31.664040 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.664043 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 22:04:31.664089 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 22:04:31.664139 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":97}
I20260925 22:04:31.664868 29486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed54e005694580b361dda5aa91cc6a" candidate_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee622e41361d49dc8209224d2a3acde5" is_pre_election: true
I20260925 22:04:31.664912 29617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed54e005694580b361dda5aa91cc6a" candidate_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85881b3516aa4f4490532cb25784f6c3" is_pre_election: true
I20260925 22:04:31.665063 29486 raft_consensus.cc:2454] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b029046c90fe4b14bfdb8c98f6ba528c in term 0.
I20260925 22:04:31.665063 29617 raft_consensus.cc:2454] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b029046c90fe4b14bfdb8c98f6ba528c in term 0.
I20260925 22:04:31.665297 29683 leader_election.cc:302] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b029046c90fe4b14bfdb8c98f6ba528c, ee622e41361d49dc8209224d2a3acde5; no voters:
I20260925 22:04:31.665485 29816 raft_consensus.cc:2781] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 22:04:31.665530 29816 raft_consensus.cc:482] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 22:04:31.665545 29816 raft_consensus.cc:3037] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:31.666388 29816 raft_consensus.cc:504] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.666525 29816 leader_election.cc:288] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [CANDIDATE]: Term 1 election: Requested vote from peers 85881b3516aa4f4490532cb25784f6c3 (127.9.112.66:45981), ee622e41361d49dc8209224d2a3acde5 (127.9.112.65:37635)
I20260925 22:04:31.666678 29617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed54e005694580b361dda5aa91cc6a" candidate_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85881b3516aa4f4490532cb25784f6c3"
I20260925 22:04:31.666774 29617 raft_consensus.cc:3037] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:31.667577 29617 raft_consensus.cc:2454] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b029046c90fe4b14bfdb8c98f6ba528c in term 1.
I20260925 22:04:31.667685 29486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed54e005694580b361dda5aa91cc6a" candidate_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee622e41361d49dc8209224d2a3acde5"
I20260925 22:04:31.667779 29684 leader_election.cc:302] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85881b3516aa4f4490532cb25784f6c3, b029046c90fe4b14bfdb8c98f6ba528c; no voters:
I20260925 22:04:31.667778 29486 raft_consensus.cc:3037] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:31.667872 29816 raft_consensus.cc:2781] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:31.668068 29816 raft_consensus.cc:686] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 1 LEADER]: Becoming Leader. State: Replica: b029046c90fe4b14bfdb8c98f6ba528c, State: Running, Role: LEADER
I20260925 22:04:31.668184 29816 consensus_queue.cc:237] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } }
I20260925 22:04:31.668743 29486 raft_consensus.cc:2454] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b029046c90fe4b14bfdb8c98f6ba528c in term 1.
I20260925 22:04:31.669171 29343 catalog_manager.cc:5495] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c reported cstate change: term changed from 0 to 1, leader changed from <none> to b029046c90fe4b14bfdb8c98f6ba528c (127.9.112.67). New cstate: current_term: 1 leader_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b029046c90fe4b14bfdb8c98f6ba528c" member_type: VOTER last_known_addr { host: "127.9.112.67" port: 40631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 } health_report { overall_health: UNKNOWN } } }
I20260925 22:04:31.684311 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.681687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39264 (local address 127.9.112.67:40631)
0925 22:04:31.681772 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:31.681774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:31.681784 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:04:31.681832 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:31.681833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:31.681854 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:31.681931 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:31.682165 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.682723 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.683549 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:31.683754 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:31.683816 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:31.683956 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:31.684095 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:31.684161 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:31.684174 (+ 13us) server_negotiation.cc:299] Negotiation successful
0925 22:04:31.684192 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 22:04:31.686740 29617 raft_consensus.cc:1261] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 1 FOLLOWER]: Refusing update from remote peer b029046c90fe4b14bfdb8c98f6ba528c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:31.686743 29486 raft_consensus.cc:1261] T 40ed54e005694580b361dda5aa91cc6a P ee622e41361d49dc8209224d2a3acde5 [term 1 FOLLOWER]: Refusing update from remote peer b029046c90fe4b14bfdb8c98f6ba528c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:31.687184 29818 consensus_queue.cc:1035] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee622e41361d49dc8209224d2a3acde5" member_type: VOTER last_known_addr { host: "127.9.112.65" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:31.687361 29816 consensus_queue.cc:1035] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85881b3516aa4f4490532cb25784f6c3" member_type: VOTER last_known_addr { host: "127.9.112.66" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:31.690390 29822 mvcc.cc:205] Tried to move back new op lower bound from 7333371378425434112 to 7333371378353799168. Current Snapshot: MvccSnapshot[applied={T|T < 7333371378425434112}]
I20260925 22:04:31.692236 29343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:31.692369 29343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 22:04:31.693219 29343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71e00014fd6143e4a6e9bec4496b2e30: Sending DeleteTablet for 3 replicas of tablet 40ed54e005694580b361dda5aa91cc6a
I20260925 22:04:31.693583 29597 tablet_service.cc:1514] Processing DeleteTablet for tablet 40ed54e005694580b361dda5aa91cc6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:31 UTC) from {username='slave'} at 127.0.0.1:48228
I20260925 22:04:31.693584 29728 tablet_service.cc:1514] Processing DeleteTablet for tablet 40ed54e005694580b361dda5aa91cc6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:31 UTC) from {username='slave'} at 127.0.0.1:39256
I20260925 22:04:31.693778 29466 tablet_service.cc:1514] Processing DeleteTablet for tablet 40ed54e005694580b361dda5aa91cc6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 22:04:31 UTC) from {username='slave'} at 127.0.0.1:33758
I20260925 22:04:31.693902 29832 tablet_replica.cc:330] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: stopping tablet replica
I20260925 22:04:31.693987 29831 tablet_replica.cc:330] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: stopping tablet replica
I20260925 22:04:31.694001 29832 raft_consensus.cc:2229] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 1 LEADER]: Raft consensus shutting down.
I20260925 22:04:31.694084 29831 raft_consensus.cc:2229] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 22:04:31.694093 29832 raft_consensus.cc:2258] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:31.694183 29831 raft_consensus.cc:2258] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 22:04:31.694401 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29404
I20260925 22:04:31.694577 29832 ts_tablet_manager.cc:1905] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:31.694607 29831 ts_tablet_manager.cc:1905] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 22:04:31.696606 29832 ts_tablet_manager.cc:1918] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:31.696617 29831 ts_tablet_manager.cc:1918] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 22:04:31.696676 29831 log.cc:1196] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-1/wal/wals/40ed54e005694580b361dda5aa91cc6a
I20260925 22:04:31.696676 29832 log.cc:1196] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Deleting WAL directory at /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790373628768857-9665-0/minicluster-data/ts-2/wal/wals/40ed54e005694580b361dda5aa91cc6a
I20260925 22:04:31.696915 29831 ts_tablet_manager.cc:1939] T 40ed54e005694580b361dda5aa91cc6a P 85881b3516aa4f4490532cb25784f6c3: Deleting consensus metadata
I20260925 22:04:31.696947 29832 ts_tablet_manager.cc:1939] T 40ed54e005694580b361dda5aa91cc6a P b029046c90fe4b14bfdb8c98f6ba528c: Deleting consensus metadata
I20260925 22:04:31.697618 29329 catalog_manager.cc:4841] TS b029046c90fe4b14bfdb8c98f6ba528c (127.9.112.67:40631): tablet 40ed54e005694580b361dda5aa91cc6a (table test-table [id=ed533feac73846279f36c082300be79f]) successfully deleted
I20260925 22:04:31.698395 29331 catalog_manager.cc:4841] TS 85881b3516aa4f4490532cb25784f6c3 (127.9.112.66:45981): tablet 40ed54e005694580b361dda5aa91cc6a (table test-table [id=ed533feac73846279f36c082300be79f]) successfully deleted
W20260925 22:04:31.702139 29330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.112.65:37635 (error 108)
I20260925 22:04:31.702353 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29535
I20260925 22:04:31.705298 29804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:31.705130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.65:37635 (local address 127.0.0.1:33766)
0925 22:04:31.705202 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:31.705250 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.65:37635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 22:04:31.705448 29330 catalog_manager.cc:4584] TS ee622e41361d49dc8209224d2a3acde5 (127.9.112.65:37635): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40ed54e005694580b361dda5aa91cc6a: Network error: Client connection negotiation failed: client connection to 127.9.112.65:37635: connect: Connection refused (error 111)
I20260925 22:04:31.708664 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29666
I20260925 22:04:31.713553 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29313
2026-09-25T22:04:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (988 ms)
[----------] 6 tests from EncryptionPolicyTest (8816 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:31 dist-test-slave-794d krb5kdc[29837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:31 dist-test-slave-794d krb5kdc[29837](info): set up 2 sockets
Sep 25 22:04:31 dist-test-slave-794d krb5kdc[29837](info): commencing operation
krb5kdc: starting...
W20260925 22:04:32.692775 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:32 dist-test-slave-794d krb5kdc[29837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:32.824630 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:32.914630 29850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:32.914883 29850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:32.914907 29850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:32.916162 29850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:32.916208 29850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:32.916221 29850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:32.916232 29850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:32.916244 29850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:32.917855 29850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:32.918138 29850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:32.918378 29850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:32.923837 29855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:32.924005 29858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:32.923869 29856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:32.923895 29850 server_base.cc:1034] running on GCE node
I20260925 22:04:32.924429 29850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:32.924750 29850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:32.925905 29850 hybrid_clock.cc:648] HybridClock initialized: now 1790373872925882 us; error 36 us; skew 500 ppm
Sep 25 22:04:32 dist-test-slave-794d krb5kdc[29837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:32.928799 29850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:32.929308 29850 webserver.cc:460] Webserver started at http://127.0.0.1:35301/ using document root <none> and password file <none>
I20260925 22:04:32.929544 29850 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:32.929586 29850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:32.929692 29850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:32.930905 29850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "de28813aff244f98b957dcac0d681fe0"
format_stamp: "Formatted at 2026-09-25 22:04:32 on dist-test-slave-794d"
server_key: "1ac87e00842e8379cc7bc9d06a17e5eb"
server_key_iv: "a06bb54ea265d76303c7f61f8f1eb11a"
server_key_version: "encryptionkey@0"
I20260925 22:04:32.931231 29850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "de28813aff244f98b957dcac0d681fe0"
format_stamp: "Formatted at 2026-09-25 22:04:32 on dist-test-slave-794d"
server_key: "1ac87e00842e8379cc7bc9d06a17e5eb"
server_key_iv: "a06bb54ea265d76303c7f61f8f1eb11a"
server_key_version: "encryptionkey@0"
I20260925 22:04:32.932657 29850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260925 22:04:32.933537 29865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:32.933724 29850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:32.933813 29850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "de28813aff244f98b957dcac0d681fe0"
format_stamp: "Formatted at 2026-09-25 22:04:32 on dist-test-slave-794d"
server_key: "1ac87e00842e8379cc7bc9d06a17e5eb"
server_key_iv: "a06bb54ea265d76303c7f61f8f1eb11a"
server_key_version: "encryptionkey@0"
I20260925 22:04:32.933877 29850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:32.946393 29850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:32.949275 29850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:32.949411 29850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:32.954202 29850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38505
I20260925 22:04:32.954253 29917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38505 every 8 connection(s)
I20260925 22:04:32.954643 29850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:32.955770 29918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:32.958565 29918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0: Bootstrap starting.
I20260925 22:04:32.959281 29918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:32.959587 29918 log.cc:824] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:32.960423 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29850
I20260925 22:04:32.960536 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:32.960567 29918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0: No bootstrap required, opened a new log
I20260925 22:04:32.960742 9665 external_mini_cluster.cc:1428] Setting key 30e2542aae04a953e651e3fa403dcfc1
I20260925 22:04:32.962911 29918 raft_consensus.cc:348] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28813aff244f98b957dcac0d681fe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38505 } }
I20260925 22:04:32.963140 29918 raft_consensus.cc:374] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:32.963176 29918 raft_consensus.cc:729] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de28813aff244f98b957dcac0d681fe0, State: Initialized, Role: FOLLOWER
I20260925 22:04:32.963347 29918 consensus_queue.cc:260] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28813aff244f98b957dcac0d681fe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38505 } }
I20260925 22:04:32.963433 29918 raft_consensus.cc:388] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:32.963461 29918 raft_consensus.cc:482] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:32.963510 29918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:32.964740 29918 raft_consensus.cc:504] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28813aff244f98b957dcac0d681fe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38505 } }
I20260925 22:04:32.964872 29918 leader_election.cc:302] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de28813aff244f98b957dcac0d681fe0; no voters:
I20260925 22:04:32.965045 29918 leader_election.cc:288] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:32.965095 29923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:32.965250 29923 raft_consensus.cc:686] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [term 1 LEADER]: Becoming Leader. State: Replica: de28813aff244f98b957dcac0d681fe0, State: Running, Role: LEADER
I20260925 22:04:32.965327 29918 sys_catalog.cc:564] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:32.965394 29923 consensus_queue.cc:237] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28813aff244f98b957dcac0d681fe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38505 } }
I20260925 22:04:32.965888 29924 sys_catalog.cc:455] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de28813aff244f98b957dcac0d681fe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28813aff244f98b957dcac0d681fe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38505 } } }
I20260925 22:04:32.965997 29924 sys_catalog.cc:458] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:32.966143 29925 sys_catalog.cc:455] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de28813aff244f98b957dcac0d681fe0. Latest consensus state: current_term: 1 leader_uuid: "de28813aff244f98b957dcac0d681fe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de28813aff244f98b957dcac0d681fe0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38505 } } }
I20260925 22:04:32.966208 29925 sys_catalog.cc:458] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:32.966748 29931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:32.967300 29931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:32.969053 29931 catalog_manager.cc:1347] Generated new cluster ID: d42eab962bfd4b07a39614c10dac8ede
I20260925 22:04:32.969101 29931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:32 dist-test-slave-794d krb5kdc[29837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:04:32.973275 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:32.961409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41105 (local address 127.0.0.1:38505)
0925 22:04:32.961648 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:32.961659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:32.961698 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:04:32.962849 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:32.962860 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:32.962938 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:32.963130 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:32.963979 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:32.965448 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:32.966657 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:32.967340 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:32.967491 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:32.970027 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:32.970032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:32.970032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:32.970045 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:32.971654 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:32.972043 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:32.972046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:32.972046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:32.972164 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:32.972402 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:32.972403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:32.972403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:32.972669 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:32.972856 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:32.972950 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 22:04:32.973062 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":129,"threads_started":1}
I20260925 22:04:32.974473 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:32.977967 29931 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:32.978760 29931 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:32.982816 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:32.975946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36661 (local address 127.0.0.1:38505)
0925 22:04:32.975986 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:32.975988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:32.976549 (+ 561us) server_negotiation.cc:407] Connection header received
0925 22:04:32.976608 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:32.976608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:32.976619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:32.976665 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:32.977256 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:32.977874 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:32.978609 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:32.978848 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:32.978907 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:32.980153 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:32.980155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:32.980155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:32.980162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:32.981514 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:32.981880 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:32.981882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:32.981882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:32.981938 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:32.982129 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:32.982130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:32.982130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:32.982243 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:32.982481 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:32.982634 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 22:04:32.982711 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:04:32.988392 29931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de28813aff244f98b957dcac0d681fe0: Generated new TSK 0
I20260925 22:04:32.988670 29931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:32.996308 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29850
W20260925 22:04:33.001099 29835 connection.cc:535] client connection to 127.0.0.1:38505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38505): Connection reset by peer (error 104)
2026-09-25T22:04:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:33 dist-test-slave-794d krb5kdc[29949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:33 dist-test-slave-794d krb5kdc[29949](info): set up 2 sockets
Sep 25 22:04:33 dist-test-slave-794d krb5kdc[29949](info): commencing operation
krb5kdc: starting...
W20260925 22:04:33.968730 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:34 dist-test-slave-794d krb5kdc[29949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:34.096199 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:34.187394 29962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:34.187664 29962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:34.187700 29962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:34.189021 29962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:34.189067 29962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:34.189086 29962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:34.189103 29962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:34.189119 29962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:34.190778 29962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37093
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:34.191075 29962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:34.191322 29962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:34.196415 29970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:34.196477 29967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:34.196460 29968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:34.196799 29962 server_base.cc:1034] running on GCE node
I20260925 22:04:34.197129 29962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:34.197474 29962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:34.198635 29962 hybrid_clock.cc:648] HybridClock initialized: now 1790373874198611 us; error 47 us; skew 500 ppm
Sep 25 22:04:34 dist-test-slave-794d krb5kdc[29949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:34.201599 29962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:34.202153 29962 webserver.cc:460] Webserver started at http://127.0.0.1:45089/ using document root <none> and password file <none>
I20260925 22:04:34.202379 29962 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:34.202432 29962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:34.202584 29962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:34.203828 29962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "c98eb22e635d44f9b04a6ff94ba7257e"
format_stamp: "Formatted at 2026-09-25 22:04:34 on dist-test-slave-794d"
server_key: "3db055e36c3f3aeb9ce9695b7c34294d"
server_key_iv: "424a5fa343922b6bb1cf39020a12b64c"
server_key_version: "encryptionkey@0"
I20260925 22:04:34.204226 29962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "c98eb22e635d44f9b04a6ff94ba7257e"
format_stamp: "Formatted at 2026-09-25 22:04:34 on dist-test-slave-794d"
server_key: "3db055e36c3f3aeb9ce9695b7c34294d"
server_key_iv: "424a5fa343922b6bb1cf39020a12b64c"
server_key_version: "encryptionkey@0"
I20260925 22:04:34.205668 29962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:34.206571 29977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:34.206722 29962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:34.206813 29962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "c98eb22e635d44f9b04a6ff94ba7257e"
format_stamp: "Formatted at 2026-09-25 22:04:34 on dist-test-slave-794d"
server_key: "3db055e36c3f3aeb9ce9695b7c34294d"
server_key_iv: "424a5fa343922b6bb1cf39020a12b64c"
server_key_version: "encryptionkey@0"
I20260925 22:04:34.206880 29962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:34.224077 29962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:34.226524 29962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:34.226681 29962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:34.231293 29962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37093
I20260925 22:04:34.231375 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37093 every 8 connection(s)
I20260925 22:04:34.231740 29962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:34.232290 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 29962
I20260925 22:04:34.232439 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:34.232735 9665 external_mini_cluster.cc:1428] Setting key 179a7fc9461510c1b6c34371561e0367
I20260925 22:04:34.233165 30030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:34.236522 30030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e: Bootstrap starting.
I20260925 22:04:34.237236 30030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:34.237625 30030 log.cc:824] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:34.238400 30030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e: No bootstrap required, opened a new log
I20260925 22:04:34.238616 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:34.233411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36863 (local address 127.0.0.1:37093)
0925 22:04:34.233606 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:34.233617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:34.233645 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:04:34.234184 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:34.234191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:34.234246 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:34.234457 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:34.234730 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:34.235702 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:34.236568 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:34.237039 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:34.237151 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:34.237896 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:34.237900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:34.237900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:34.238084 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:34.238119 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:34.238131 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:34.238203 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:34.238299 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 22:04:34.238330 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260925 22:04:34.239641 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:34.240803 30030 raft_consensus.cc:348] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260925 22:04:34.240978 30030 raft_consensus.cc:374] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:34.241012 30030 raft_consensus.cc:729] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98eb22e635d44f9b04a6ff94ba7257e, State: Initialized, Role: FOLLOWER
I20260925 22:04:34.241145 30030 consensus_queue.cc:260] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260925 22:04:34.241245 30030 raft_consensus.cc:388] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:34.241326 30030 raft_consensus.cc:482] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:34.241381 30030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:34.242012 30030 raft_consensus.cc:504] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260925 22:04:34.242116 30030 leader_election.cc:302] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c98eb22e635d44f9b04a6ff94ba7257e; no voters:
I20260925 22:04:34.242323 30030 leader_election.cc:288] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:34.242362 30040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:34.242519 30040 raft_consensus.cc:686] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [term 1 LEADER]: Becoming Leader. State: Replica: c98eb22e635d44f9b04a6ff94ba7257e, State: Running, Role: LEADER
I20260925 22:04:34.242606 30030 sys_catalog.cc:564] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:34.242659 30040 consensus_queue.cc:237] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260925 22:04:34.243043 30040 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c98eb22e635d44f9b04a6ff94ba7257e. Latest consensus state: current_term: 1 leader_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } } }
I20260925 22:04:34.243088 30042 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98eb22e635d44f9b04a6ff94ba7257e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } } }
I20260925 22:04:34.243180 30040 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:34.243326 30042 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:34.243786 30047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:34.244513 30047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:34.244854 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:34.241216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45205 (local address 127.0.0.1:37093)
0925 22:04:34.241256 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:34.241258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:34.241386 (+ 128us) server_negotiation.cc:407] Connection header received
0925 22:04:34.241478 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:34.241479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:34.241490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:34.241535 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:34.241678 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:34.242196 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:34.243344 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:34.243602 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:34.243671 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:34.244530 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:34.244531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:34.244532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:34.244539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:34.244557 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:34.244562 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:34.244606 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:34.244760 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 22:04:34.244774 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:04:34.246507 30047 catalog_manager.cc:1347] Generated new cluster ID: c320ce497f434bccba284197dba5b150
I20260925 22:04:34.246568 30047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:34.252554 30047 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:34.253257 30047 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:34.260785 30047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c98eb22e635d44f9b04a6ff94ba7257e: Generated new TSK 0
I20260925 22:04:34.261102 30047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:34.267524 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 29962
W20260925 22:04:34.272310 29947 connection.cc:535] client connection to 127.0.0.1:37093 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37093): Connection reset by peer (error 104)
2026-09-25T22:04:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:34 dist-test-slave-794d krb5kdc[30061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:34 dist-test-slave-794d krb5kdc[30061](info): set up 2 sockets
Sep 25 22:04:34 dist-test-slave-794d krb5kdc[30061](info): commencing operation
krb5kdc: starting...
W20260925 22:04:35.252683 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:35 dist-test-slave-794d krb5kdc[30061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:35.381106 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:35.474864 30074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:35.475159 30074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:35.475198 30074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:35.476598 30074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:35.476658 30074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:35.476677 30074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:35.476696 30074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:35.476709 30074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:35.478456 30074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33601
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:35.478775 30074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:35.479048 30074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:35.484642 30080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:35.484629 30082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:35.484737 30074 server_base.cc:1034] running on GCE node
W20260925 22:04:35.484642 30079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:35.485195 30074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:35.485562 30074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:35.486742 30074 hybrid_clock.cc:648] HybridClock initialized: now 1790373875486717 us; error 35 us; skew 500 ppm
Sep 25 22:04:35 dist-test-slave-794d krb5kdc[30061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:35.489718 30074 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:35.490252 30074 webserver.cc:460] Webserver started at http://127.0.0.1:35637/ using document root <none> and password file <none>
I20260925 22:04:35.490523 30074 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:35.490576 30074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:35.490716 30074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:35.491947 30074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "ff0852a56501463585a88b9d2aec3f6d"
format_stamp: "Formatted at 2026-09-25 22:04:35 on dist-test-slave-794d"
server_key: "4d8015bdac5c9bd8d6d769df040e0176"
server_key_iv: "b6f68853368b0ee4245b7091bb3c4d17"
server_key_version: "encryptionkey@0"
I20260925 22:04:35.492319 30074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "ff0852a56501463585a88b9d2aec3f6d"
format_stamp: "Formatted at 2026-09-25 22:04:35 on dist-test-slave-794d"
server_key: "4d8015bdac5c9bd8d6d769df040e0176"
server_key_iv: "b6f68853368b0ee4245b7091bb3c4d17"
server_key_version: "encryptionkey@0"
I20260925 22:04:35.493759 30074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:35.494643 30089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:35.494858 30074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:35.494954 30074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "ff0852a56501463585a88b9d2aec3f6d"
format_stamp: "Formatted at 2026-09-25 22:04:35 on dist-test-slave-794d"
server_key: "4d8015bdac5c9bd8d6d769df040e0176"
server_key_iv: "b6f68853368b0ee4245b7091bb3c4d17"
server_key_version: "encryptionkey@0"
I20260925 22:04:35.495023 30074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:35.498711 30074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:35.501055 30074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:35.501189 30074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:35.505528 30074 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33601
I20260925 22:04:35.505571 30141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33601 every 8 connection(s)
I20260925 22:04:35.505903 30074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:35.506989 30142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:35.507177 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30074
I20260925 22:04:35.507282 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:35.507473 9665 external_mini_cluster.cc:1428] Setting key 67aa3f978676b1f2fcfd43f52e242b5c
I20260925 22:04:35.510237 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:35.508006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40985 (local address 127.0.0.1:33601)
0925 22:04:35.508188 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:35.508197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:35.508225 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:04:35.508830 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:35.508839 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:35.508868 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:35.509013 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:35.509739 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:35.509743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:35.509743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:35.509747 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:35.509792 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:35.509804 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:35.509867 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:35.509942 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 22:04:35.510030 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260925 22:04:35.510918 30142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d: Bootstrap starting.
I20260925 22:04:35.511462 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:35.511698 30142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:35.512027 30142 log.cc:824] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:35.513095 30142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d: No bootstrap required, opened a new log
I20260925 22:04:35.514744 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:35.513526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50489 (local address 127.0.0.1:33601)
0925 22:04:35.513567 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:35.513569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:35.513641 (+ 72us) server_negotiation.cc:407] Connection header received
0925 22:04:35.513723 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:35.513724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:35.513728 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:35.513773 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:35.514507 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:35.514508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:35.514509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:35.514513 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:35.514529 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:35.514533 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:35.514597 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:35.514663 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 22:04:35.514676 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:04:35.515681 30142 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0852a56501463585a88b9d2aec3f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33601 } }
I20260925 22:04:35.515865 30142 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:35.515890 30142 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff0852a56501463585a88b9d2aec3f6d, State: Initialized, Role: FOLLOWER
I20260925 22:04:35.516016 30142 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0852a56501463585a88b9d2aec3f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33601 } }
I20260925 22:04:35.516099 30142 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:35.516124 30142 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:35.516158 30142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:35.516893 30142 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0852a56501463585a88b9d2aec3f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33601 } }
I20260925 22:04:35.516993 30142 leader_election.cc:302] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff0852a56501463585a88b9d2aec3f6d; no voters:
I20260925 22:04:35.517195 30142 leader_election.cc:288] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:35.517331 30152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:35.517474 30142 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:35.517482 30152 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [term 1 LEADER]: Becoming Leader. State: Replica: ff0852a56501463585a88b9d2aec3f6d, State: Running, Role: LEADER
I20260925 22:04:35.517681 30152 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0852a56501463585a88b9d2aec3f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33601 } }
I20260925 22:04:35.518714 30154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff0852a56501463585a88b9d2aec3f6d. Latest consensus state: current_term: 1 leader_uuid: "ff0852a56501463585a88b9d2aec3f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0852a56501463585a88b9d2aec3f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33601 } } }
I20260925 22:04:35.518714 30153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff0852a56501463585a88b9d2aec3f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0852a56501463585a88b9d2aec3f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33601 } } }
I20260925 22:04:35.518901 30154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:35.518905 30153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:35.519143 30162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:35.519761 30162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:35.521709 30162 catalog_manager.cc:1347] Generated new cluster ID: 4d4ce876f051423989704da11b7b0269
I20260925 22:04:35.521766 30162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:35.533373 30162 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:35.533612 30162 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:35.537137 30162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff0852a56501463585a88b9d2aec3f6d: Generated new TSK 0
I20260925 22:04:35.537365 30162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:35.547022 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30074
2026-09-25T22:04:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:35 dist-test-slave-794d krb5kdc[30173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:35 dist-test-slave-794d krb5kdc[30173](info): set up 2 sockets
Sep 25 22:04:35 dist-test-slave-794d krb5kdc[30173](info): commencing operation
krb5kdc: starting...
W20260925 22:04:36.537336 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:36 dist-test-slave-794d krb5kdc[30173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:36.680279 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:36.776571 30186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:36.776840 30186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:36.776876 30186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:36.778210 30186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:36.778267 30186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:36.778285 30186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:36.778304 30186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:36.778317 30186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:36.780010 30186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:36.780306 30186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:36.780593 30186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:04:36.788019 30186 server_base.cc:1034] running on GCE node
W20260925 22:04:36.787947 30191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:36.787993 30194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:36.788192 30192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:36.788527 30186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:36.788899 30186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:36.790175 30186 hybrid_clock.cc:648] HybridClock initialized: now 1790373876790130 us; error 38 us; skew 500 ppm
Sep 25 22:04:36 dist-test-slave-794d krb5kdc[30173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:36.794148 30186 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:36.795135 30186 webserver.cc:460] Webserver started at http://127.0.0.1:34667/ using document root <none> and password file <none>
I20260925 22:04:36.795436 30186 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:36.795490 30186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:36.795609 30186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:36.797009 30186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551"
format_stamp: "Formatted at 2026-09-25 22:04:36 on dist-test-slave-794d"
server_key: "d2f89c844b53ca94a8a43ed19a14b8ad"
server_key_iv: "f21e9fd9b694115bfb17415309031d13"
server_key_version: "encryptionkey@0"
I20260925 22:04:36.797396 30186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551"
format_stamp: "Formatted at 2026-09-25 22:04:36 on dist-test-slave-794d"
server_key: "d2f89c844b53ca94a8a43ed19a14b8ad"
server_key_iv: "f21e9fd9b694115bfb17415309031d13"
server_key_version: "encryptionkey@0"
I20260925 22:04:36.798880 30186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 22:04:36.799854 30201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:36.800120 30186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:36.800222 30186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551"
format_stamp: "Formatted at 2026-09-25 22:04:36 on dist-test-slave-794d"
server_key: "d2f89c844b53ca94a8a43ed19a14b8ad"
server_key_iv: "f21e9fd9b694115bfb17415309031d13"
server_key_version: "encryptionkey@0"
I20260925 22:04:36.800293 30186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:36.828749 30186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:36.831660 30186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:36.831843 30186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:36.836512 30186 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45125
I20260925 22:04:36.836544 30253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45125 every 8 connection(s)
I20260925 22:04:36.836988 30186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:36.838119 30254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:36.840816 30254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551: Bootstrap starting.
I20260925 22:04:36.841503 30254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:36.841799 30254 log.cc:824] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:36.842641 30254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551: No bootstrap required, opened a new log
I20260925 22:04:36.844549 30254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260925 22:04:36.844718 30254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:36.844743 30254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c2fcf54f3a452f8ab2c5f6bf22c551, State: Initialized, Role: FOLLOWER
I20260925 22:04:36.844907 30254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260925 22:04:36.844980 30254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:36.845002 30254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:36.845036 30254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:36.845636 30254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260925 22:04:36.845732 30254 leader_election.cc:302] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87c2fcf54f3a452f8ab2c5f6bf22c551; no voters:
I20260925 22:04:36.845932 30254 leader_election.cc:288] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:36.846012 30257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:36.846227 30257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [term 1 LEADER]: Becoming Leader. State: Replica: 87c2fcf54f3a452f8ab2c5f6bf22c551, State: Running, Role: LEADER
I20260925 22:04:36.846247 30254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:36.846446 30257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260925 22:04:36.847134 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30186
I20260925 22:04:36.847257 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:36.847390 30258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } } }
I20260925 22:04:36.847407 30259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87c2fcf54f3a452f8ab2c5f6bf22c551. Latest consensus state: current_term: 1 leader_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2fcf54f3a452f8ab2c5f6bf22c551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } } }
I20260925 22:04:36.847479 9665 external_mini_cluster.cc:1428] Setting key f8d2b6ae6179e0be828e14fbb03e9287
I20260925 22:04:36.847491 30259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:36.847484 30258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:36.847776 30270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:36.848448 30270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:36.850270 30270 catalog_manager.cc:1347] Generated new cluster ID: e65726c712b6489fb84d1625b417fc08
I20260925 22:04:36.850318 30270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:36 dist-test-slave-794d krb5kdc[30173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:04:36.862382 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:36.848254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38861 (local address 127.0.0.1:45125)
0925 22:04:36.848716 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:36.848727 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:36.848765 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:04:36.848893 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:36.848901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:36.848986 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:36.849148 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:36.849980 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:36.851615 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:36.852822 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:36.853619 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:36.853727 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:36.857238 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:36.857246 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:36.857246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:36.857265 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:36.859789 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:36.860261 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:36.860266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:36.860267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:36.860433 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:36.860653 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:36.860653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:36.860654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:36.861052 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:36.861227 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:36.861762 (+ 535us) server_negotiation.cc:299] Negotiation successful
0925 22:04:36.862059 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":107,"threads_started":1}
I20260925 22:04:36.863066 30270 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:36.863690 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:36.863931 30270 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:36.873893 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:36.865590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40879 (local address 127.0.0.1:45125)
0925 22:04:36.865633 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:36.865635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:36.865754 (+ 119us) server_negotiation.cc:407] Connection header received
0925 22:04:36.865825 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:36.865826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:36.865839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:36.865894 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:36.866681 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:36.867566 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:36.868773 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:36.869124 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:36.869198 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:36.870910 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:36.870911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:36.870912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:36.870921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:36.872661 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:36.873071 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:36.873072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:36.873073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:36.873142 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:36.873329 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:36.873330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:36.873330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:36.873476 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:36.873593 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:36.873682 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 22:04:36.873774 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 22:04:36.881858 30270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87c2fcf54f3a452f8ab2c5f6bf22c551: Generated new TSK 0
I20260925 22:04:36.882121 30270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:36.886301 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30186
2026-09-25T22:04:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:36 dist-test-slave-794d krb5kdc[30285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:36 dist-test-slave-794d krb5kdc[30285](info): set up 2 sockets
Sep 25 22:04:36 dist-test-slave-794d krb5kdc[30285](info): commencing operation
krb5kdc: starting...
W20260925 22:04:37.865005 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:37 dist-test-slave-794d krb5kdc[30285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:37.996687 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:38.086941 30298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:38.087178 30298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:38.087201 30298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:38.088467 30298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:38.088513 30298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:38.088526 30298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:38.088537 30298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:38.088547 30298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:38.090080 30298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:38.090351 30298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:38.090641 30298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:38.095958 30306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:38.095952 30303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:38.095952 30304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:38.096385 30298 server_base.cc:1034] running on GCE node
I20260925 22:04:38.096657 30298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:38.096976 30298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:38.098145 30298 hybrid_clock.cc:648] HybridClock initialized: now 1790373878098132 us; error 50 us; skew 500 ppm
Sep 25 22:04:38 dist-test-slave-794d krb5kdc[30285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:38.109485 30298 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:38.110097 30298 webserver.cc:460] Webserver started at http://127.0.0.1:32985/ using document root <none> and password file <none>
I20260925 22:04:38.110402 30298 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:38.110464 30298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:38.110651 30298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:38.111905 30298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "cac5effa41644078b9a66639f173cb4a"
format_stamp: "Formatted at 2026-09-25 22:04:38 on dist-test-slave-794d"
server_key: "a0e832d1c678fce8fcfef0219d02f466"
server_key_iv: "626d109c931eafa656bf20d57d50e9ac"
server_key_version: "encryptionkey@0"
I20260925 22:04:38.112278 30298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "cac5effa41644078b9a66639f173cb4a"
format_stamp: "Formatted at 2026-09-25 22:04:38 on dist-test-slave-794d"
server_key: "a0e832d1c678fce8fcfef0219d02f466"
server_key_iv: "626d109c931eafa656bf20d57d50e9ac"
server_key_version: "encryptionkey@0"
I20260925 22:04:38.113758 30298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:38.114668 30313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:38.114887 30298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:38.114990 30298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "cac5effa41644078b9a66639f173cb4a"
format_stamp: "Formatted at 2026-09-25 22:04:38 on dist-test-slave-794d"
server_key: "a0e832d1c678fce8fcfef0219d02f466"
server_key_iv: "626d109c931eafa656bf20d57d50e9ac"
server_key_version: "encryptionkey@0"
I20260925 22:04:38.115069 30298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:38.162084 30298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:38.164754 30298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:38.165000 30298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:38.169858 30298 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35881
I20260925 22:04:38.169898 30365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35881 every 8 connection(s)
I20260925 22:04:38.170301 30298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:38.171523 30366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:38.173583 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30298
I20260925 22:04:38.173712 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:38.173909 9665 external_mini_cluster.cc:1428] Setting key 8ac218fbec52d6c2d6d4da0bb728de4c
I20260925 22:04:38.174675 30366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a: Bootstrap starting.
I20260925 22:04:38.175349 30366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:38.175637 30366 log.cc:824] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:38.176396 30366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a: No bootstrap required, opened a new log
I20260925 22:04:38.178321 30366 raft_consensus.cc:348] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5effa41644078b9a66639f173cb4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35881 } }
I20260925 22:04:38.178486 30366 raft_consensus.cc:374] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:38.178535 30366 raft_consensus.cc:729] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac5effa41644078b9a66639f173cb4a, State: Initialized, Role: FOLLOWER
I20260925 22:04:38.178665 30366 consensus_queue.cc:260] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5effa41644078b9a66639f173cb4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35881 } }
I20260925 22:04:38.178750 30366 raft_consensus.cc:388] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:38.178788 30366 raft_consensus.cc:482] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:38.178838 30366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:38.179410 30366 raft_consensus.cc:504] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5effa41644078b9a66639f173cb4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35881 } }
I20260925 22:04:38.179523 30366 leader_election.cc:302] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cac5effa41644078b9a66639f173cb4a; no voters:
I20260925 22:04:38.179673 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:38.174577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40829 (local address 127.0.0.1:35881)
0925 22:04:38.174777 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:38.174788 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:38.174829 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:04:38.175311 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:38.175319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:38.175370 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:38.175520 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:38.175758 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:38.176898 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:38.177684 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:38.178119 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:38.178217 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:38.178974 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:38.178979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:38.178979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:38.179165 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:38.179199 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:38.179210 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:38.179261 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:38.179368 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 22:04:38.179440 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260925 22:04:38.179716 30366 leader_election.cc:288] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:38.179747 30371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:38.179862 30371 raft_consensus.cc:686] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [term 1 LEADER]: Becoming Leader. State: Replica: cac5effa41644078b9a66639f173cb4a, State: Running, Role: LEADER
I20260925 22:04:38.179967 30371 consensus_queue.cc:237] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5effa41644078b9a66639f173cb4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35881 } }
I20260925 22:04:38.179996 30366 sys_catalog.cc:564] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:38.180426 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cac5effa41644078b9a66639f173cb4a. Latest consensus state: current_term: 1 leader_uuid: "cac5effa41644078b9a66639f173cb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5effa41644078b9a66639f173cb4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35881 } } }
I20260925 22:04:38.180644 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:38.180905 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:38.180583 30372 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cac5effa41644078b9a66639f173cb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac5effa41644078b9a66639f173cb4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35881 } } }
I20260925 22:04:38.181102 30372 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:38.181242 30376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:38.181753 30376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:38.184736 30376 catalog_manager.cc:1347] Generated new cluster ID: 370e26e39739449cb0bc9c0abf1c0b45
I20260925 22:04:38.184793 30376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:38.187956 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:38.182824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46445 (local address 127.0.0.1:35881)
0925 22:04:38.182865 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:38.182867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:38.184499 (+ 1632us) server_negotiation.cc:407] Connection header received
0925 22:04:38.184559 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:38.184560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:38.184571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:38.184625 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:38.184766 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:38.185534 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:38.186580 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:38.186897 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:38.186979 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:38.187700 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:38.187702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:38.187702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:38.187707 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:38.187726 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:38.187731 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:38.187787 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:38.187847 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 22:04:38.187875 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:04:38.197372 30376 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:38.198098 30376 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:38.201630 30376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cac5effa41644078b9a66639f173cb4a: Generated new TSK 0
I20260925 22:04:38.201843 30376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:38.206653 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30298
2026-09-25T22:04:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:38 dist-test-slave-794d krb5kdc[30397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:38 dist-test-slave-794d krb5kdc[30397](info): set up 2 sockets
Sep 25 22:04:38 dist-test-slave-794d krb5kdc[30397](info): commencing operation
krb5kdc: starting...
W20260925 22:04:39.176762 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:39 dist-test-slave-794d krb5kdc[30397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:39.312469 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:39.406590 30410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:39.406876 30410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:39.406909 30410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:39.408247 30410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:39.408299 30410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:39.408313 30410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:39.408325 30410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:39.408360 30410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:39.410167 30410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:39.410494 30410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:39.410753 30410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:39.416160 30418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:39.416162 30416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:39.416168 30415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:39.416466 30410 server_base.cc:1034] running on GCE node
I20260925 22:04:39.416780 30410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:39.417109 30410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:39.418262 30410 hybrid_clock.cc:648] HybridClock initialized: now 1790373879418242 us; error 37 us; skew 500 ppm
Sep 25 22:04:39 dist-test-slave-794d krb5kdc[30397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:39.420974 30410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:39.421427 30410 webserver.cc:460] Webserver started at http://127.0.0.1:39807/ using document root <none> and password file <none>
I20260925 22:04:39.421656 30410 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:39.421697 30410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:39.421806 30410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:39.423099 30410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "7514bafcc84749c2b40912772d733785"
format_stamp: "Formatted at 2026-09-25 22:04:39 on dist-test-slave-794d"
server_key: "be9bfa6f628208a4f3fd7d6c3180d8ab"
server_key_iv: "7f2bf341b075b4611daf89266ce2b5d1"
server_key_version: "encryptionkey@0"
I20260925 22:04:39.423520 30410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "7514bafcc84749c2b40912772d733785"
format_stamp: "Formatted at 2026-09-25 22:04:39 on dist-test-slave-794d"
server_key: "be9bfa6f628208a4f3fd7d6c3180d8ab"
server_key_iv: "7f2bf341b075b4611daf89266ce2b5d1"
server_key_version: "encryptionkey@0"
I20260925 22:04:39.424966 30410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 22:04:39.425788 30425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:39.425997 30410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:39.426103 30410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "7514bafcc84749c2b40912772d733785"
format_stamp: "Formatted at 2026-09-25 22:04:39 on dist-test-slave-794d"
server_key: "be9bfa6f628208a4f3fd7d6c3180d8ab"
server_key_iv: "7f2bf341b075b4611daf89266ce2b5d1"
server_key_version: "encryptionkey@0"
I20260925 22:04:39.426178 30410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:39.429976 30410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:39.432511 30410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:39.432669 30410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:39.437520 30410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39241
I20260925 22:04:39.437559 30477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39241 every 8 connection(s)
I20260925 22:04:39.437929 30410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:39.438287 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30410
I20260925 22:04:39.438512 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:39.438827 9665 external_mini_cluster.cc:1428] Setting key 94b1d04548a8228ed9d757461baaf281
I20260925 22:04:39.439349 30478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:39.441737 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:39.439451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50849 (local address 127.0.0.1:39241)
0925 22:04:39.439639 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:39.439648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:39.439677 (+ 29us) server_negotiation.cc:407] Connection header received
0925 22:04:39.440369 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:39.440381 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:39.440419 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:39.440567 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:39.441279 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:39.441283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:39.441284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:39.441287 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:39.441332 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:39.441344 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:39.441400 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:39.441474 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 22:04:39.441519 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260925 22:04:39.442714 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:39.443204 30478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785: Bootstrap starting.
I20260925 22:04:39.444036 30478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:39.444528 30478 log.cc:824] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:39.445470 30478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785: No bootstrap required, opened a new log
I20260925 22:04:39.445833 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:39.444629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46993 (local address 127.0.0.1:39241)
0925 22:04:39.444670 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:39.444672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:39.444799 (+ 127us) server_negotiation.cc:407] Connection header received
0925 22:04:39.444907 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:39.444908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:39.444914 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:39.444975 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:39.445623 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:39.445624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:39.445625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:39.445629 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:39.445648 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:39.445652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:39.445698 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:39.445756 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 22:04:39.445768 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:04:39.448184 30478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514bafcc84749c2b40912772d733785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260925 22:04:39.448519 30478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:39.448583 30478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7514bafcc84749c2b40912772d733785, State: Initialized, Role: FOLLOWER
I20260925 22:04:39.448730 30478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514bafcc84749c2b40912772d733785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260925 22:04:39.448864 30478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:39.448915 30478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:39.448971 30478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:39.449681 30478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514bafcc84749c2b40912772d733785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260925 22:04:39.449887 30478 leader_election.cc:302] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7514bafcc84749c2b40912772d733785; no voters:
I20260925 22:04:39.450115 30478 leader_election.cc:288] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:39.450160 30488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:39.450338 30488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [term 1 LEADER]: Becoming Leader. State: Replica: 7514bafcc84749c2b40912772d733785, State: Running, Role: LEADER
I20260925 22:04:39.450488 30488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514bafcc84749c2b40912772d733785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260925 22:04:39.450837 30478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:39.451004 30489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7514bafcc84749c2b40912772d733785" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514bafcc84749c2b40912772d733785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } } }
I20260925 22:04:39.451016 30490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7514bafcc84749c2b40912772d733785. Latest consensus state: current_term: 1 leader_uuid: "7514bafcc84749c2b40912772d733785" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514bafcc84749c2b40912772d733785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } } }
I20260925 22:04:39.451143 30490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:39.451367 30489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:39.451381 30492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:39.452452 30492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:39.454366 30492 catalog_manager.cc:1347] Generated new cluster ID: db716fab33904d43a74e0016e2626161
I20260925 22:04:39.454442 30492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:39.467666 30492 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:39.467888 30492 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:39.473420 30492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7514bafcc84749c2b40912772d733785: Generated new TSK 0
I20260925 22:04:39.473716 30492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:39.480132 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30410
2026-09-25T22:04:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:39 dist-test-slave-794d krb5kdc[30509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:39 dist-test-slave-794d krb5kdc[30509](info): set up 2 sockets
Sep 25 22:04:39 dist-test-slave-794d krb5kdc[30509](info): commencing operation
krb5kdc: starting...
W20260925 22:04:40.452816 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:40 dist-test-slave-794d krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:40.584108 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40683
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:40.671979 30522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:40.672228 30522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:40.672261 30522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:40.673566 30522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:40.673624 30522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:40.673650 30522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:40.673666 30522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:40.673681 30522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:40.675335 30522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:40.675652 30522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:40.675901 30522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:40.681243 30528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:40.681270 30527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:40.681269 30530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:40.681434 30522 server_base.cc:1034] running on GCE node
I20260925 22:04:40.681787 30522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:40.682106 30522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:40.683267 30522 hybrid_clock.cc:648] HybridClock initialized: now 1790373880683241 us; error 48 us; skew 500 ppm
Sep 25 22:04:40 dist-test-slave-794d krb5kdc[30509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:40.686115 30522 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:40.686580 30522 webserver.cc:460] Webserver started at http://127.0.0.1:37843/ using document root <none> and password file <none>
I20260925 22:04:40.686810 30522 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:40.686868 30522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:40.687005 30522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:40.688181 30522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "455ec211d00b46f4a824bd768b381cd7"
format_stamp: "Formatted at 2026-09-25 22:04:40 on dist-test-slave-794d"
server_key: "18c29839e3d258750c7a74451571856e"
server_key_iv: "6e58f60aee268069538f0732b8d53266"
server_key_version: "encryptionkey@0"
I20260925 22:04:40.688560 30522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "455ec211d00b46f4a824bd768b381cd7"
format_stamp: "Formatted at 2026-09-25 22:04:40 on dist-test-slave-794d"
server_key: "18c29839e3d258750c7a74451571856e"
server_key_iv: "6e58f60aee268069538f0732b8d53266"
server_key_version: "encryptionkey@0"
I20260925 22:04:40.690047 30522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:40.690981 30537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:40.691215 30522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:40.691341 30522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "455ec211d00b46f4a824bd768b381cd7"
format_stamp: "Formatted at 2026-09-25 22:04:40 on dist-test-slave-794d"
server_key: "18c29839e3d258750c7a74451571856e"
server_key_iv: "6e58f60aee268069538f0732b8d53266"
server_key_version: "encryptionkey@0"
I20260925 22:04:40.691430 30522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:40.714155 30522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:40.716917 30522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:40.717060 30522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:40.721586 30522 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40683
I20260925 22:04:40.721622 30589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40683 every 8 connection(s)
I20260925 22:04:40.722011 30522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:40.723109 30590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:40.725839 30590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7: Bootstrap starting.
I20260925 22:04:40.726512 30590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:40.726802 30590 log.cc:824] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:40.727682 30590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7: No bootstrap required, opened a new log
I20260925 22:04:40.729488 30590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ec211d00b46f4a824bd768b381cd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40683 } }
I20260925 22:04:40.729648 30590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:40.729672 30590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455ec211d00b46f4a824bd768b381cd7, State: Initialized, Role: FOLLOWER
I20260925 22:04:40.729832 30590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ec211d00b46f4a824bd768b381cd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40683 } }
I20260925 22:04:40.729911 30590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:40.729947 30590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:40.730001 30590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:40.730032 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30522
I20260925 22:04:40.730163 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:40.730402 9665 external_mini_cluster.cc:1428] Setting key 32e8b213c9f8725f26505e6f3f5baf44
I20260925 22:04:40.730711 30590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ec211d00b46f4a824bd768b381cd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40683 } }
I20260925 22:04:40.730845 30590 leader_election.cc:302] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 455ec211d00b46f4a824bd768b381cd7; no voters:
I20260925 22:04:40.731216 30590 leader_election.cc:288] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:40.731249 30594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:40.731477 30594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [term 1 LEADER]: Becoming Leader. State: Replica: 455ec211d00b46f4a824bd768b381cd7, State: Running, Role: LEADER
I20260925 22:04:40.731635 30590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:40.731629 30594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ec211d00b46f4a824bd768b381cd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40683 } }
I20260925 22:04:40.732100 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "455ec211d00b46f4a824bd768b381cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ec211d00b46f4a824bd768b381cd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40683 } } }
I20260925 22:04:40.732319 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:40.732260 30596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 455ec211d00b46f4a824bd768b381cd7. Latest consensus state: current_term: 1 leader_uuid: "455ec211d00b46f4a824bd768b381cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ec211d00b46f4a824bd768b381cd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40683 } } }
I20260925 22:04:40.732627 30596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:40.732641 30604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:40.733321 30604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:40.737843 30604 catalog_manager.cc:1347] Generated new cluster ID: ce382c337d264dd5a400356836a1dbd3
I20260925 22:04:40.737897 30604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:40 dist-test-slave-794d krb5kdc[30509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:04:40.743166 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:40.731214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:59167 (local address 127.0.0.1:40683)
0925 22:04:40.731485 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:40.731497 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:40.731533 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:04:40.731673 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:40.731682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:40.731753 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:40.731914 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:40.732839 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:40.734213 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:40.735022 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:40.735473 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:40.735576 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:40.738567 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:40.738571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:40.738572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:40.738586 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:40.741265 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:40.741736 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:40.741739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:40.741739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:40.741870 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:40.742186 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:40.742186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:40.742187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:40.742499 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:40.742651 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:40.742792 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 22:04:40.742894 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
I20260925 22:04:40.744413 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:40.753667 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:40.746134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:60425 (local address 127.0.0.1:40683)
0925 22:04:40.746174 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:40.746176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:40.746289 (+ 113us) server_negotiation.cc:407] Connection header received
0925 22:04:40.746368 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:40.746369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:40.746378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:40.746402 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:40.747059 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:40.747847 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:40.748862 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:40.749264 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:40.749318 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:40.750876 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:40.750878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:40.750878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:40.750885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:40.752421 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:40.752851 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:40.752852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:40.752853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:40.752922 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:40.753135 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:40.753135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:40.753136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:40.753270 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:40.753386 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:40.753499 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 22:04:40.753560 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:04:40.762225 30604 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:40.763015 30604 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:40.765826 30604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 455ec211d00b46f4a824bd768b381cd7: Generated new TSK 0
I20260925 22:04:40.766062 30604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:40.772320 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30522
W20260925 22:04:40.777163 30507 connection.cc:535] client connection to 127.0.0.1:40683 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40683): Connection reset by peer (error 104)
2026-09-25T22:04:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:40 dist-test-slave-794d krb5kdc[30621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:40 dist-test-slave-794d krb5kdc[30621](info): set up 2 sockets
Sep 25 22:04:40 dist-test-slave-794d krb5kdc[30621](info): commencing operation
krb5kdc: starting...
W20260925 22:04:41.744702 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:41 dist-test-slave-794d krb5kdc[30621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:41.872406 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:41.962527 30634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:41.962810 30634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:41.962839 30634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:41.964184 30634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:41.964242 30634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:41.964257 30634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:41.964268 30634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:41.964279 30634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:41.965986 30634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:41.966279 30634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:41.966537 30634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:41.972026 30640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:41.972154 30642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:41.972237 30639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:41.973230 30634 server_base.cc:1034] running on GCE node
I20260925 22:04:41.973599 30634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:41.973982 30634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:41.975163 30634 hybrid_clock.cc:648] HybridClock initialized: now 1790373881975145 us; error 39 us; skew 500 ppm
Sep 25 22:04:41 dist-test-slave-794d krb5kdc[30621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:41.978255 30634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:41.979009 30634 webserver.cc:460] Webserver started at http://127.0.0.1:34699/ using document root <none> and password file <none>
I20260925 22:04:41.979264 30634 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:41.979316 30634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:41.979454 30634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:41.980787 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "d6a218fc49894a9dae8a376350f60b6b"
format_stamp: "Formatted at 2026-09-25 22:04:41 on dist-test-slave-794d"
server_key: "90a9dc0c786173591ca9828ce9907537"
server_key_iv: "8a1fd0d53d8bf7aab37c32d2e556377a"
server_key_version: "encryptionkey@0"
I20260925 22:04:41.981163 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "d6a218fc49894a9dae8a376350f60b6b"
format_stamp: "Formatted at 2026-09-25 22:04:41 on dist-test-slave-794d"
server_key: "90a9dc0c786173591ca9828ce9907537"
server_key_iv: "8a1fd0d53d8bf7aab37c32d2e556377a"
server_key_version: "encryptionkey@0"
I20260925 22:04:41.982823 30634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:41.984066 30649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:41.984272 30634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:41.984424 30634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "d6a218fc49894a9dae8a376350f60b6b"
format_stamp: "Formatted at 2026-09-25 22:04:41 on dist-test-slave-794d"
server_key: "90a9dc0c786173591ca9828ce9907537"
server_key_iv: "8a1fd0d53d8bf7aab37c32d2e556377a"
server_key_version: "encryptionkey@0"
I20260925 22:04:41.984503 30634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:42.003798 30634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:42.006541 30634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:42.006712 30634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:42.011871 30634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34749
I20260925 22:04:42.012015 30701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34749 every 8 connection(s)
I20260925 22:04:42.012394 30634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:42.013432 30702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:42.016067 30702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b: Bootstrap starting.
I20260925 22:04:42.016798 30702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:42.017063 30702 log.cc:824] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:42.017907 30702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b: No bootstrap required, opened a new log
I20260925 22:04:42.018468 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30634
I20260925 22:04:42.018589 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:42.018790 9665 external_mini_cluster.cc:1428] Setting key ba83f626524b59733683a8a6c3ba5f1d
I20260925 22:04:42.020260 30702 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a218fc49894a9dae8a376350f60b6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260925 22:04:42.020653 30702 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:42.020721 30702 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a218fc49894a9dae8a376350f60b6b, State: Initialized, Role: FOLLOWER
I20260925 22:04:42.021013 30702 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a218fc49894a9dae8a376350f60b6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260925 22:04:42.021163 30702 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:42.021202 30702 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:42.021260 30702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:42.022209 30702 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a218fc49894a9dae8a376350f60b6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260925 22:04:42.022330 30702 leader_election.cc:302] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6a218fc49894a9dae8a376350f60b6b; no voters:
I20260925 22:04:42.022534 30702 leader_election.cc:288] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:42.022605 30707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:42.022763 30707 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [term 1 LEADER]: Becoming Leader. State: Replica: d6a218fc49894a9dae8a376350f60b6b, State: Running, Role: LEADER
I20260925 22:04:42.022825 30702 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:42.022914 30707 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a218fc49894a9dae8a376350f60b6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260925 22:04:42.024274 30709 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6a218fc49894a9dae8a376350f60b6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a218fc49894a9dae8a376350f60b6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } } }
I20260925 22:04:42.024413 30709 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:42.024271 30708 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6a218fc49894a9dae8a376350f60b6b. Latest consensus state: current_term: 1 leader_uuid: "d6a218fc49894a9dae8a376350f60b6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a218fc49894a9dae8a376350f60b6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } } }
I20260925 22:04:42.024998 30708 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b [sys.catalog]: This master's current role is: LEADER
W20260925 22:04:42.025139 30721 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:42.025187 30721 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:42.025292 30723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:42.025481 30723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:42.026317 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:42.019497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:42673 (local address 127.0.0.1:34749)
0925 22:04:42.019746 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:42.019759 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:42.019800 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:04:42.020211 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:42.020220 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:42.020296 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:42.020567 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:42.021071 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:42.022833 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:42.023941 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:42.024641 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:42.024756 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:42.025612 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:42.025618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:42.025618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:42.025816 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:42.025854 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:42.025867 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:42.025937 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:42.026054 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 22:04:42.026084 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":116,"threads_started":1}
I20260925 22:04:42.027397 30723 catalog_manager.cc:1347] Generated new cluster ID: 2c0b2316bf594d149f3c592a213d6b59
I20260925 22:04:42.027453 30723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:42.027616 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:42.032368 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:42.029313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:55681 (local address 127.0.0.1:34749)
0925 22:04:42.029356 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:42.029357 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:42.029484 (+ 127us) server_negotiation.cc:407] Connection header received
0925 22:04:42.029548 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:42.029549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:42.029572 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:42.029620 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:42.029764 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:42.030288 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:42.031017 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:42.031304 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:42.031364 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:42.032087 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:42.032088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:42.032089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:42.032093 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:42.032113 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:42.032118 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:42.032166 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:42.032234 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 22:04:42.032249 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 22:04:42.039151 30723 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:42.040009 30723 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:42.051096 30723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6a218fc49894a9dae8a376350f60b6b: Generated new TSK 0
I20260925 22:04:42.051419 30723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:42.053547 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30634
W20260925 22:04:42.058315 30619 connection.cc:535] client connection to 127.0.0.1:34749 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34749): Connection reset by peer (error 104)
2026-09-25T22:04:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:42 dist-test-slave-794d krb5kdc[30733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:42 dist-test-slave-794d krb5kdc[30733](info): set up 2 sockets
Sep 25 22:04:42 dist-test-slave-794d krb5kdc[30733](info): commencing operation
krb5kdc: starting...
W20260925 22:04:43.056734 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:43 dist-test-slave-794d krb5kdc[30733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:43.184566 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:43.276371 30746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:43.276655 30746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:43.276682 30746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:43.277973 30746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:43.278025 30746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:43.278038 30746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:43.278050 30746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:43.278060 30746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:43.279670 30746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:43.279933 30746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:43.280143 30746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:43.285583 30752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:43.285597 30751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:43.285600 30754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:43.285872 30746 server_base.cc:1034] running on GCE node
I20260925 22:04:43.286160 30746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:43.286473 30746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:43.287654 30746 hybrid_clock.cc:648] HybridClock initialized: now 1790373883287620 us; error 54 us; skew 500 ppm
Sep 25 22:04:43 dist-test-slave-794d krb5kdc[30733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:43.294584 30746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:43.295091 30746 webserver.cc:460] Webserver started at http://127.0.0.1:43207/ using document root <none> and password file <none>
I20260925 22:04:43.295331 30746 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:43.295379 30746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:43.295511 30746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:43.296777 30746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "bd448f0af54342c9a323a8365b65d6e6"
format_stamp: "Formatted at 2026-09-25 22:04:43 on dist-test-slave-794d"
server_key: "5a3a3dcf559dc44009e9ae5bcc452309"
server_key_iv: "9c421abd852d7c6f6c8872cc5deb1e6e"
server_key_version: "encryptionkey@0"
I20260925 22:04:43.297149 30746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "bd448f0af54342c9a323a8365b65d6e6"
format_stamp: "Formatted at 2026-09-25 22:04:43 on dist-test-slave-794d"
server_key: "5a3a3dcf559dc44009e9ae5bcc452309"
server_key_iv: "9c421abd852d7c6f6c8872cc5deb1e6e"
server_key_version: "encryptionkey@0"
I20260925 22:04:43.298570 30746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:43.299328 30761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:43.299535 30746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:43.299621 30746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "bd448f0af54342c9a323a8365b65d6e6"
format_stamp: "Formatted at 2026-09-25 22:04:43 on dist-test-slave-794d"
server_key: "5a3a3dcf559dc44009e9ae5bcc452309"
server_key_iv: "9c421abd852d7c6f6c8872cc5deb1e6e"
server_key_version: "encryptionkey@0"
I20260925 22:04:43.299693 30746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:43.303243 30746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:43.305656 30746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:43.305796 30746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:43.310348 30746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39897
I20260925 22:04:43.310405 30813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39897 every 8 connection(s)
I20260925 22:04:43.310768 30746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:43.311848 30814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:43.314754 30814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6: Bootstrap starting.
I20260925 22:04:43.315519 30814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:43.315788 30814 log.cc:824] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:43.316617 30814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6: No bootstrap required, opened a new log
I20260925 22:04:43.318416 30814 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd448f0af54342c9a323a8365b65d6e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 22:04:43.318599 30814 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:43.318624 30814 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd448f0af54342c9a323a8365b65d6e6, State: Initialized, Role: FOLLOWER
I20260925 22:04:43.318759 30814 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd448f0af54342c9a323a8365b65d6e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 22:04:43.318849 30814 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:43.318883 30814 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:43.318921 30814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:43.319536 30814 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd448f0af54342c9a323a8365b65d6e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 22:04:43.319634 30814 leader_election.cc:302] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd448f0af54342c9a323a8365b65d6e6; no voters:
I20260925 22:04:43.319793 30814 leader_election.cc:288] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:43.319928 30817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:43.320087 30814 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:43.320111 30817 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [term 1 LEADER]: Becoming Leader. State: Replica: bd448f0af54342c9a323a8365b65d6e6, State: Running, Role: LEADER
I20260925 22:04:43.320369 30817 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd448f0af54342c9a323a8365b65d6e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 22:04:43.320504 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30746
I20260925 22:04:43.320632 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:43.320878 9665 external_mini_cluster.cc:1428] Setting key 701017e57fb7ee6a23c38471e66f0923
I20260925 22:04:43.321450 30818 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd448f0af54342c9a323a8365b65d6e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd448f0af54342c9a323a8365b65d6e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260925 22:04:43.321578 30818 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:43.321450 30819 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd448f0af54342c9a323a8365b65d6e6. Latest consensus state: current_term: 1 leader_uuid: "bd448f0af54342c9a323a8365b65d6e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd448f0af54342c9a323a8365b65d6e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260925 22:04:43.321885 30833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:43.322083 30819 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:43.322635 30833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:43.323868 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:43.321786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:56259 (local address 127.0.0.1:39897)
0925 22:04:43.322062 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:43.322074 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:43.322113 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:04:43.322259 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:43.322265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:43.322294 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:43.322454 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:43.323401 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:43.323405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:43.323405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:43.323410 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:43.323461 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:43.323477 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:43.323548 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:43.323624 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 22:04:43.323670 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260925 22:04:43.324550 30833 catalog_manager.cc:1347] Generated new cluster ID: 496a1029927449498431a11d0af66cad
I20260925 22:04:43.324602 30833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:43.324929 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:43.327934 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:43.326718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:60115 (local address 127.0.0.1:39897)
0925 22:04:43.326827 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:43.326829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:43.326951 (+ 122us) server_negotiation.cc:407] Connection header received
0925 22:04:43.327016 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:43.327017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:43.327024 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:43.327077 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:43.327700 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:43.327702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:43.327702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:43.327706 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:43.327722 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:43.327726 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:43.327768 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:43.327822 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 22:04:43.327841 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 22:04:43.342154 30833 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:43.342406 30833 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:43.344571 30833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd448f0af54342c9a323a8365b65d6e6: Generated new TSK 0
I20260925 22:04:43.344786 30833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:43.348421 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30746
2026-09-25T22:04:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:43 dist-test-slave-794d krb5kdc[30845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:43 dist-test-slave-794d krb5kdc[30845](info): set up 2 sockets
Sep 25 22:04:43 dist-test-slave-794d krb5kdc[30845](info): commencing operation
krb5kdc: starting...
W20260925 22:04:44.316864 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:44 dist-test-slave-794d krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:44.448858 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:44.544775 30858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:44.545053 30858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:44.545077 30858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:44.546402 30858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:44.546463 30858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:44.546482 30858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:44.546500 30858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:44.546516 30858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:44.548646 30858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:44.548919 30858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:44.549139 30858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:44.554528 30863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:44.554584 30858 server_base.cc:1034] running on GCE node
W20260925 22:04:44.554543 30866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:44.554559 30864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:44.555161 30858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:44.555512 30858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:44.556663 30858 hybrid_clock.cc:648] HybridClock initialized: now 1790373884556645 us; error 37 us; skew 500 ppm
Sep 25 22:04:44 dist-test-slave-794d krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:44.563359 30858 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:44.563961 30858 webserver.cc:460] Webserver started at http://127.0.0.1:40009/ using document root <none> and password file <none>
I20260925 22:04:44.564203 30858 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:44.564253 30858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:44.564435 30858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:44.565662 30858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "4b02714b56b14add9f50e29a6ced1b6d"
format_stamp: "Formatted at 2026-09-25 22:04:44 on dist-test-slave-794d"
server_key: "1fce3bd34fa03d757c1e874acdc4d66c"
server_key_iv: "40a5e5df742e043c1e343db7664fffc9"
server_key_version: "encryptionkey@0"
I20260925 22:04:44.566022 30858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "4b02714b56b14add9f50e29a6ced1b6d"
format_stamp: "Formatted at 2026-09-25 22:04:44 on dist-test-slave-794d"
server_key: "1fce3bd34fa03d757c1e874acdc4d66c"
server_key_iv: "40a5e5df742e043c1e343db7664fffc9"
server_key_version: "encryptionkey@0"
I20260925 22:04:44.567502 30858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:44.568369 30873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:44.568548 30858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:44.568640 30858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "4b02714b56b14add9f50e29a6ced1b6d"
format_stamp: "Formatted at 2026-09-25 22:04:44 on dist-test-slave-794d"
server_key: "1fce3bd34fa03d757c1e874acdc4d66c"
server_key_iv: "40a5e5df742e043c1e343db7664fffc9"
server_key_version: "encryptionkey@0"
I20260925 22:04:44.568711 30858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:44.581012 30858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:44.583534 30858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:44.583707 30858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:44.589095 30858 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42161
I20260925 22:04:44.589165 30925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42161 every 8 connection(s)
I20260925 22:04:44.589593 30858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:44.590768 30926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:44.593660 30926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d: Bootstrap starting.
I20260925 22:04:44.594385 30926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:44.594699 30926 log.cc:824] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:44.595124 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30858
I20260925 22:04:44.595223 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:44.595422 9665 external_mini_cluster.cc:1428] Setting key 35e411f9658a175f5634ad60e7eefc46
I20260925 22:04:44.595712 30926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d: No bootstrap required, opened a new log
I20260925 22:04:44.598376 30926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b02714b56b14add9f50e29a6ced1b6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42161 } }
I20260925 22:04:44.598572 30926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:44.598603 30926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b02714b56b14add9f50e29a6ced1b6d, State: Initialized, Role: FOLLOWER
I20260925 22:04:44.598735 30926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b02714b56b14add9f50e29a6ced1b6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42161 } }
I20260925 22:04:44.598812 30926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:44.598838 30926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:44.598896 30926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:44.599607 30926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b02714b56b14add9f50e29a6ced1b6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42161 } }
I20260925 22:04:44.599709 30926 leader_election.cc:302] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b02714b56b14add9f50e29a6ced1b6d; no voters:
I20260925 22:04:44.599900 30926 leader_election.cc:288] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:44.599943 30931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:44.600114 30931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [term 1 LEADER]: Becoming Leader. State: Replica: 4b02714b56b14add9f50e29a6ced1b6d, State: Running, Role: LEADER
I20260925 22:04:44.600232 30926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:44.600216 30931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b02714b56b14add9f50e29a6ced1b6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42161 } }
I20260925 22:04:44.600637 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b02714b56b14add9f50e29a6ced1b6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b02714b56b14add9f50e29a6ced1b6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42161 } } }
I20260925 22:04:44.600796 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:44.600983 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b02714b56b14add9f50e29a6ced1b6d. Latest consensus state: current_term: 1 leader_uuid: "4b02714b56b14add9f50e29a6ced1b6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b02714b56b14add9f50e29a6ced1b6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42161 } } }
I20260925 22:04:44.601097 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:44.601224 30938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:44.601835 30938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:44.603551 30938 catalog_manager.cc:1347] Generated new cluster ID: 323d7de0797a44a095d95713cb73e2c5
I20260925 22:04:44.603602 30938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:44 dist-test-slave-794d krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 22:04:44.608958 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:44.596149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:49437 (local address 127.0.0.1:42161)
0925 22:04:44.596330 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:44.596366 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:44.596404 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:04:44.597295 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:44.597307 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:44.597376 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:44.597540 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:44.598446 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:44.599980 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:44.601081 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:44.601728 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:44.601840 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:44.604914 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:44.604918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:44.604919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:44.604933 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:44.607045 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:44.607538 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:44.607541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:44.607542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:44.607667 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:44.607887 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:44.607888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:44.607888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:44.608251 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:44.608475 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:44.608577 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 22:04:44.608695 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260925 22:04:44.610167 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:44.619719 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:44.612036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:34367 (local address 127.0.0.1:42161)
0925 22:04:44.612103 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:44.612114 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:44.612367 (+ 253us) server_negotiation.cc:407] Connection header received
0925 22:04:44.612455 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:44.612456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:44.612468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:44.612522 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:44.613161 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:44.613967 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:44.615045 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:44.615376 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:44.615433 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:44.616952 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:44.616953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:44.616953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:44.616960 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:44.618517 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:44.618931 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:44.618931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:44.618932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:44.618990 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:44.619197 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:44.619197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:44.619198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:44.619318 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:44.619421 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:44.619552 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:04:44.619618 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 22:04:44.626039 30938 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:44.626616 30938 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:44.635645 30938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b02714b56b14add9f50e29a6ced1b6d: Generated new TSK 0
I20260925 22:04:44.635948 30938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:44.642050 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30858
W20260925 22:04:44.646621 30843 connection.cc:535] client connection to 127.0.0.1:42161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42161): Connection reset by peer (error 104)
2026-09-25T22:04:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:44 dist-test-slave-794d krb5kdc[30957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:44 dist-test-slave-794d krb5kdc[30957](info): set up 2 sockets
Sep 25 22:04:44 dist-test-slave-794d krb5kdc[30957](info): commencing operation
krb5kdc: starting...
W20260925 22:04:45.612993 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:45 dist-test-slave-794d krb5kdc[30957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:45.752815 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:45.856204 30970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:45.856534 30970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:45.856568 30970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:45.857928 30970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:45.857980 30970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:45.857999 30970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:45.858014 30970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:45.858028 30970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:45.859715 30970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:45.860013 30970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:45.860260 30970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:45.865828 30978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:45.865847 30975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:45.865895 30976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:45.866007 30970 server_base.cc:1034] running on GCE node
I20260925 22:04:45.866501 30970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:45.866840 30970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:45.868006 30970 hybrid_clock.cc:648] HybridClock initialized: now 1790373885867967 us; error 57 us; skew 500 ppm
Sep 25 22:04:45 dist-test-slave-794d krb5kdc[30957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:45.871148 30970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:45.871687 30970 webserver.cc:460] Webserver started at http://127.0.0.1:46767/ using document root <none> and password file <none>
I20260925 22:04:45.871934 30970 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:45.871986 30970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:45.872130 30970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:45.873322 30970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c"
format_stamp: "Formatted at 2026-09-25 22:04:45 on dist-test-slave-794d"
server_key: "ae94a4ffe0214cb9c7e16cbab5c7e693"
server_key_iv: "342bd6bca69177192a4847530e39e825"
server_key_version: "encryptionkey@0"
I20260925 22:04:45.873688 30970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c"
format_stamp: "Formatted at 2026-09-25 22:04:45 on dist-test-slave-794d"
server_key: "ae94a4ffe0214cb9c7e16cbab5c7e693"
server_key_iv: "342bd6bca69177192a4847530e39e825"
server_key_version: "encryptionkey@0"
I20260925 22:04:45.875192 30970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:45.876132 30985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:45.876472 30970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:45.876587 30970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c"
format_stamp: "Formatted at 2026-09-25 22:04:45 on dist-test-slave-794d"
server_key: "ae94a4ffe0214cb9c7e16cbab5c7e693"
server_key_iv: "342bd6bca69177192a4847530e39e825"
server_key_version: "encryptionkey@0"
I20260925 22:04:45.876667 30970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:45.894769 30970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:45.897997 30970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:45.898216 30970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:45.902678 30970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33141
I20260925 22:04:45.902742 31037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33141 every 8 connection(s)
I20260925 22:04:45.903129 30970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:45.904250 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:45.907056 31038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c: Bootstrap starting.
I20260925 22:04:45.907683 31038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:45.907963 31038 log.cc:824] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:45.908775 31038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c: No bootstrap required, opened a new log
I20260925 22:04:45.909386 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 30970
I20260925 22:04:45.909571 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:45.909791 9665 external_mini_cluster.cc:1428] Setting key 84be8ed5ca0b6693edcb46909fedccb9
I20260925 22:04:45.910861 31038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33141 } }
I20260925 22:04:45.911079 31038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:45.911116 31038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74a8ebdcd95b419abc9e37bdb35b7d1c, State: Initialized, Role: FOLLOWER
I20260925 22:04:45.911260 31038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33141 } }
I20260925 22:04:45.911351 31038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:45.911392 31038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:45.911442 31038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:45.912541 31038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33141 } }
I20260925 22:04:45.912676 31038 leader_election.cc:302] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74a8ebdcd95b419abc9e37bdb35b7d1c; no voters:
I20260925 22:04:45.912917 31038 leader_election.cc:288] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:45.912962 31043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:45.913203 31043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [term 1 LEADER]: Becoming Leader. State: Replica: 74a8ebdcd95b419abc9e37bdb35b7d1c, State: Running, Role: LEADER
I20260925 22:04:45.913301 31038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:45.913466 31043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33141 } }
I20260925 22:04:45.914038 31044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33141 } } }
I20260925 22:04:45.914194 31044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:45.914366 31044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74a8ebdcd95b419abc9e37bdb35b7d1c. Latest consensus state: current_term: 1 leader_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74a8ebdcd95b419abc9e37bdb35b7d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33141 } } }
I20260925 22:04:45.914441 31044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:45.914602 31052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:45.915211 31052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:45.916455 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:45.910593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:51523 (local address 127.0.0.1:33141)
0925 22:04:45.910888 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:45.910901 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:45.910942 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:04:45.911070 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:45.911079 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:45.911148 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:45.911315 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:45.911763 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:45.912951 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:45.914147 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:45.914762 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:45.914877 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:45.915703 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:45.915708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:45.915708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:45.915908 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:45.915947 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:45.915960 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:45.916035 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:45.916138 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 22:04:45.916170 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":163,"threads_started":1}
I20260925 22:04:45.917481 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:45.919348 31052 catalog_manager.cc:1347] Generated new cluster ID: 41ecb98841a74a77990d51077b4541a8
I20260925 22:04:45.919413 31052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:45.923378 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:45.919059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:55615 (local address 127.0.0.1:33141)
0925 22:04:45.919104 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:45.919107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:45.919556 (+ 449us) server_negotiation.cc:407] Connection header received
0925 22:04:45.919598 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:45.919599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:45.919612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:45.919685 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:45.919852 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:45.920734 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:45.921821 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:45.922165 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:45.922259 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:45.923048 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:45.923049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:45.923050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:45.923055 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:45.923078 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:45.923083 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:45.923153 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:45.923244 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 22:04:45.923266 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 22:04:45.937901 31052 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:45.938535 31052 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:45.944417 31052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74a8ebdcd95b419abc9e37bdb35b7d1c: Generated new TSK 0
I20260925 22:04:45.944617 31052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:45.953792 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 30970
W20260925 22:04:45.958038 30955 connection.cc:535] client connection to 127.0.0.1:33141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33141): Connection reset by peer (error 104)
2026-09-25T22:04:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:46 dist-test-slave-794d krb5kdc[31069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:46 dist-test-slave-794d krb5kdc[31069](info): set up 2 sockets
Sep 25 22:04:46 dist-test-slave-794d krb5kdc[31069](info): commencing operation
krb5kdc: starting...
W20260925 22:04:46.924827 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:47 dist-test-slave-794d krb5kdc[31069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 22:04:47.068794 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45321
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:47.158074 31082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:47.158342 31082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:47.158365 31082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:47.159654 31082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:47.159708 31082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:47.159729 31082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:47.159741 31082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:47.159752 31082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:47.161406 31082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45321
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:47.161703 31082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:47.161972 31082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:47.167497 31088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:47.167491 31087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:47.167552 31082 server_base.cc:1034] running on GCE node
W20260925 22:04:47.167491 31090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:47.168057 31082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:47.168421 31082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:47.169612 31082 hybrid_clock.cc:648] HybridClock initialized: now 1790373887169581 us; error 50 us; skew 500 ppm
Sep 25 22:04:47 dist-test-slave-794d krb5kdc[31069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:47.172621 31082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 22:04:47.173138 31082 webserver.cc:460] Webserver started at http://127.0.0.1:39171/ using document root <none> and password file <none>
I20260925 22:04:47.173378 31082 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:47.173431 31082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:47.173576 31082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:47.174817 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "a4ab747e40b640fc84ee51efe13c2339"
format_stamp: "Formatted at 2026-09-25 22:04:47 on dist-test-slave-794d"
server_key: "49160891ed83264a10fc4e2e5740b4cc"
server_key_iv: "e4e494038372151febda519589bdef47"
server_key_version: "encryptionkey@0"
I20260925 22:04:47.175210 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "a4ab747e40b640fc84ee51efe13c2339"
format_stamp: "Formatted at 2026-09-25 22:04:47 on dist-test-slave-794d"
server_key: "49160891ed83264a10fc4e2e5740b4cc"
server_key_iv: "e4e494038372151febda519589bdef47"
server_key_version: "encryptionkey@0"
I20260925 22:04:47.176661 31082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:47.177454 31097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:47.177632 31082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:47.177721 31082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "a4ab747e40b640fc84ee51efe13c2339"
format_stamp: "Formatted at 2026-09-25 22:04:47 on dist-test-slave-794d"
server_key: "49160891ed83264a10fc4e2e5740b4cc"
server_key_iv: "e4e494038372151febda519589bdef47"
server_key_version: "encryptionkey@0"
I20260925 22:04:47.177784 31082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:47.181324 31082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:47.183584 31082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:47.183722 31082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:47.188293 31082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45321
I20260925 22:04:47.188360 31149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45321 every 8 connection(s)
I20260925 22:04:47.188742 31082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:47.189926 31150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:47.192539 31150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339: Bootstrap starting.
I20260925 22:04:47.193188 31150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:47.193485 31150 log.cc:824] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:47.194314 31150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339: No bootstrap required, opened a new log
I20260925 22:04:47.195191 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31082
I20260925 22:04:47.195318 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:47.195502 9665 external_mini_cluster.cc:1428] Setting key 633c22bbc7a90c603ad664047d6a9ee6
I20260925 22:04:47.196448 31150 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ab747e40b640fc84ee51efe13c2339" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45321 } }
I20260925 22:04:47.196662 31150 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:47.196703 31150 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ab747e40b640fc84ee51efe13c2339, State: Initialized, Role: FOLLOWER
I20260925 22:04:47.196844 31150 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ab747e40b640fc84ee51efe13c2339" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45321 } }
I20260925 22:04:47.196920 31150 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:47.196954 31150 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:47.197022 31150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:47.198194 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:47.196423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:50261 (local address 127.0.0.1:45321)
0925 22:04:47.196653 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:47.196665 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:47.196704 (+ 39us) server_negotiation.cc:407] Connection header received
0925 22:04:47.196842 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:47.196858 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:47.196887 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:47.197030 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:47.197748 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:47.197751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:47.197751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:47.197755 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:47.197809 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:47.197821 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:47.197881 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:47.197948 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 22:04:47.197988 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260925 22:04:47.198182 31150 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ab747e40b640fc84ee51efe13c2339" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45321 } }
I20260925 22:04:47.198328 31150 leader_election.cc:302] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4ab747e40b640fc84ee51efe13c2339; no voters:
I20260925 22:04:47.198513 31150 leader_election.cc:288] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:47.198557 31155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:47.198695 31155 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [term 1 LEADER]: Becoming Leader. State: Replica: a4ab747e40b640fc84ee51efe13c2339, State: Running, Role: LEADER
I20260925 22:04:47.198787 31150 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:47.199397 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:47.200279 31155 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ab747e40b640fc84ee51efe13c2339" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45321 } }
I20260925 22:04:47.201241 31156 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4ab747e40b640fc84ee51efe13c2339. Latest consensus state: current_term: 1 leader_uuid: "a4ab747e40b640fc84ee51efe13c2339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ab747e40b640fc84ee51efe13c2339" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45321 } } }
I20260925 22:04:47.201332 31156 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:47.201543 31175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:47.201761 31157 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4ab747e40b640fc84ee51efe13c2339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ab747e40b640fc84ee51efe13c2339" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45321 } } }
I20260925 22:04:47.201974 31157 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:47.202363 31175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:47.202688 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:47.201151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.64:45327 (local address 127.0.0.1:45321)
0925 22:04:47.201193 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:47.201195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:47.201482 (+ 287us) server_negotiation.cc:407] Connection header received
0925 22:04:47.201563 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:47.201564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:47.201570 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:47.201627 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:47.202462 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:47.202463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:47.202464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:47.202468 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:47.202486 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:47.202491 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:47.202519 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:47.202612 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 22:04:47.202629 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:04:47.204306 31175 catalog_manager.cc:1347] Generated new cluster ID: 269076a1fbed4855a69e17379ec39787
I20260925 22:04:47.204396 31175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:47.217504 31175 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:47.217743 31175 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:47.224709 31175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4ab747e40b640fc84ee51efe13c2339: Generated new TSK 0
I20260925 22:04:47.224957 31175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:47.227002 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31082
2026-09-25T22:04:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:47 dist-test-slave-794d krb5kdc[31181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:47 dist-test-slave-794d krb5kdc[31181](info): set up 2 sockets
Sep 25 22:04:47 dist-test-slave-794d krb5kdc[31181](info): commencing operation
krb5kdc: starting...
W20260925 22:04:48.196969 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:48 dist-test-slave-794d krb5kdc[31181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:48.328272 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35171
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:35171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:48.417953 31194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:48.418226 31194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:48.418251 31194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:48.419551 31194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:48.419598 31194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:48.419612 31194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:48.419624 31194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:48.419634 31194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:48.421346 31194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:35171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:35171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:48.421626 31194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:48.421885 31194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:04:48.428027 31194 server_base.cc:1034] running on GCE node
W20260925 22:04:48.427971 31202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:48.427992 31199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:48.427973 31200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:48.428519 31194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:48.428848 31194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:48.430023 31194 hybrid_clock.cc:648] HybridClock initialized: now 1790373888430006 us; error 37 us; skew 500 ppm
Sep 25 22:04:48 dist-test-slave-794d krb5kdc[31181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373888, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:48.436146 31194 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:48.436677 31194 webserver.cc:460] Webserver started at http://127.9.112.126:44519/ using document root <none> and password file <none>
I20260925 22:04:48.436915 31194 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:48.436955 31194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:48.437062 31194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:48.438207 31194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "210f5c3be67849618e2ab92c1ffced1b"
format_stamp: "Formatted at 2026-09-25 22:04:48 on dist-test-slave-794d"
server_key: "0b0d238b9ae2b59b9eea1625477b7598"
server_key_iv: "29e817ebac489bc630360580e4bddb0f"
server_key_version: "encryptionkey@0"
I20260925 22:04:48.438542 31194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "210f5c3be67849618e2ab92c1ffced1b"
format_stamp: "Formatted at 2026-09-25 22:04:48 on dist-test-slave-794d"
server_key: "0b0d238b9ae2b59b9eea1625477b7598"
server_key_iv: "29e817ebac489bc630360580e4bddb0f"
server_key_version: "encryptionkey@0"
I20260925 22:04:48.439903 31194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:48.440714 31209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:48.440922 31194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:48.441007 31194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "210f5c3be67849618e2ab92c1ffced1b"
format_stamp: "Formatted at 2026-09-25 22:04:48 on dist-test-slave-794d"
server_key: "0b0d238b9ae2b59b9eea1625477b7598"
server_key_iv: "29e817ebac489bc630360580e4bddb0f"
server_key_version: "encryptionkey@0"
I20260925 22:04:48.441067 31194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:48.450273 31194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:48.452485 31194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:48.452607 31194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:48.456986 31194 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:35171
I20260925 22:04:48.457067 31261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:35171 every 8 connection(s)
I20260925 22:04:48.457423 31194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:48.458439 31262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:48.461114 31262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b: Bootstrap starting.
I20260925 22:04:48.461758 31262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:48.462028 31262 log.cc:824] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:48.462740 31262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b: No bootstrap required, opened a new log
I20260925 22:04:48.464011 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31194
I20260925 22:04:48.464114 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:48.464311 9665 external_mini_cluster.cc:1428] Setting key 212709a1b0c89fb1b4c03c0f6d515fb2
I20260925 22:04:48.464672 31262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f5c3be67849618e2ab92c1ffced1b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35171 } }
I20260925 22:04:48.464880 31262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:48.464953 31262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210f5c3be67849618e2ab92c1ffced1b, State: Initialized, Role: FOLLOWER
I20260925 22:04:48.465132 31262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f5c3be67849618e2ab92c1ffced1b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35171 } }
I20260925 22:04:48.465246 31262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:48.465308 31262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:48.465386 31262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:48.466665 31262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f5c3be67849618e2ab92c1ffced1b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35171 } }
I20260925 22:04:48.466868 31262 leader_election.cc:302] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210f5c3be67849618e2ab92c1ffced1b; no voters:
I20260925 22:04:48.467036 31262 leader_election.cc:288] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:48.467093 31267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:48.467315 31267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [term 1 LEADER]: Becoming Leader. State: Replica: 210f5c3be67849618e2ab92c1ffced1b, State: Running, Role: LEADER
I20260925 22:04:48.467375 31262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:48.467545 31267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f5c3be67849618e2ab92c1ffced1b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35171 } }
I20260925 22:04:48.468255 31268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210f5c3be67849618e2ab92c1ffced1b. Latest consensus state: current_term: 1 leader_uuid: "210f5c3be67849618e2ab92c1ffced1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f5c3be67849618e2ab92c1ffced1b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35171 } } }
I20260925 22:04:48.468364 31268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:48.468560 31269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210f5c3be67849618e2ab92c1ffced1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f5c3be67849618e2ab92c1ffced1b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 35171 } } }
I20260925 22:04:48.468667 31269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:48.468832 31276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:48.469430 31276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:48.471433 31276 catalog_manager.cc:1347] Generated new cluster ID: 13d7c635e3af46e8a0d87cd2b32bfb82
I20260925 22:04:48.471491 31276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:48 dist-test-slave-794d krb5kdc[31181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:48.477310 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:48.465117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43787 (local address 127.9.112.126:35171)
0925 22:04:48.465409 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:48.465421 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:48.465458 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:04:48.465599 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:48.465608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:48.465668 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:48.465843 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:48.466664 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:48.468042 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:48.469356 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:48.470021 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:48.470123 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:48.473298 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:48.473303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:48.473303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:48.473320 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:48.475102 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:48.475544 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:48.475549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:48.475549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:48.475669 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:48.475891 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:48.475892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:48.475892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:48.476499 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:48.476825 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:48.476970 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 22:04:48.477091 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":104,"threads_started":1}
I20260925 22:04:48.478382 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:48.481281 31276 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:48.482033 31276 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:48.485924 31276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210f5c3be67849618e2ab92c1ffced1b: Generated new TSK 0
I20260925 22:04:48.486138 31276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:48.491622 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:48.480697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54897 (local address 127.9.112.126:35171)
0925 22:04:48.480957 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:48.480959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:48.480972 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:04:48.480997 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:48.480998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:48.481009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:48.481069 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:48.484939 (+ 3870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:48.485476 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:48.487044 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:48.487288 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:48.487353 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:48.488927 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:48.488929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:48.488929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:48.488935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:48.490424 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:48.490868 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:48.490869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:48.490869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:48.490933 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:48.491143 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:48.491144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:48.491144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:48.491260 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:48.491355 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:48.491486 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 22:04:48.491546 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260925 22:04:48.494099 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31194
W20260925 22:04:48.499146 31179 connection.cc:535] client connection to 127.9.112.126:35171 recv error: Network error: failed to read from TLS socket (remote: 127.9.112.126:35171): Connection reset by peer (error 104)
2026-09-25T22:04:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:48 dist-test-slave-794d krb5kdc[31293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:48 dist-test-slave-794d krb5kdc[31293](info): set up 2 sockets
Sep 25 22:04:48 dist-test-slave-794d krb5kdc[31293](info): commencing operation
krb5kdc: starting...
W20260925 22:04:49.460798 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:49 dist-test-slave-794d krb5kdc[31293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:49.595566 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:43615
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:33185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:43615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:49.688059 31306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:49.688367 31306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:49.688403 31306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:49.689692 31306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:49.689739 31306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:49.689757 31306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:49.689781 31306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:49.689797 31306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:49.691411 31306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:43615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:43615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:49.691706 31306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:49.691960 31306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:49.697324 31311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:49.697335 31314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:49.697335 31312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:49.697559 31306 server_base.cc:1034] running on GCE node
I20260925 22:04:49.697794 31306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:49.698128 31306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:49.699306 31306 hybrid_clock.cc:648] HybridClock initialized: now 1790373889699263 us; error 57 us; skew 500 ppm
Sep 25 22:04:49 dist-test-slave-794d krb5kdc[31293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373889, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:49.702584 31306 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:49.703132 31306 webserver.cc:460] Webserver started at http://127.9.112.126:44245/ using document root <none> and password file <none>
I20260925 22:04:49.703374 31306 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:49.703428 31306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:49.703559 31306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:49.704857 31306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "eed19806e0aa49c2b137ec0fe750aa09"
format_stamp: "Formatted at 2026-09-25 22:04:49 on dist-test-slave-794d"
server_key: "402cd5138702a56f69c0dced944adeeb"
server_key_iv: "5f8fb192cfecb472c3064d7a20350371"
server_key_version: "encryptionkey@0"
I20260925 22:04:49.705226 31306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "eed19806e0aa49c2b137ec0fe750aa09"
format_stamp: "Formatted at 2026-09-25 22:04:49 on dist-test-slave-794d"
server_key: "402cd5138702a56f69c0dced944adeeb"
server_key_iv: "5f8fb192cfecb472c3064d7a20350371"
server_key_version: "encryptionkey@0"
I20260925 22:04:49.706796 31306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:49.707579 31321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:49.707752 31306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:49.707842 31306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "eed19806e0aa49c2b137ec0fe750aa09"
format_stamp: "Formatted at 2026-09-25 22:04:49 on dist-test-slave-794d"
server_key: "402cd5138702a56f69c0dced944adeeb"
server_key_iv: "5f8fb192cfecb472c3064d7a20350371"
server_key_version: "encryptionkey@0"
I20260925 22:04:49.707908 31306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:49.728546 31306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:49.731088 31306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:49.731256 31306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:49.737203 31306 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:43615
I20260925 22:04:49.737293 31373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:43615 every 8 connection(s)
I20260925 22:04:49.737751 31306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:49.738806 31374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:49.742043 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31306
I20260925 22:04:49.742144 31374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09: Bootstrap starting.
I20260925 22:04:49.742170 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:49.742404 9665 external_mini_cluster.cc:1428] Setting key 6a06ff39ad288f4543eaf6c7be60f4c1
I20260925 22:04:49.743021 31374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:49.743362 31374 log.cc:824] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:49.744172 31374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09: No bootstrap required, opened a new log
I20260925 22:04:49.745882 31374 raft_consensus.cc:348] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed19806e0aa49c2b137ec0fe750aa09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43615 } }
I20260925 22:04:49.746095 31374 raft_consensus.cc:374] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:49.746134 31374 raft_consensus.cc:729] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed19806e0aa49c2b137ec0fe750aa09, State: Initialized, Role: FOLLOWER
I20260925 22:04:49.746280 31374 consensus_queue.cc:260] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed19806e0aa49c2b137ec0fe750aa09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43615 } }
I20260925 22:04:49.746364 31374 raft_consensus.cc:388] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:49.746404 31374 raft_consensus.cc:482] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:49.746462 31374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:49.747275 31374 raft_consensus.cc:504] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed19806e0aa49c2b137ec0fe750aa09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43615 } }
I20260925 22:04:49.747377 31374 leader_election.cc:302] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eed19806e0aa49c2b137ec0fe750aa09; no voters:
I20260925 22:04:49.747561 31374 leader_election.cc:288] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:49.747746 31380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:49.747921 31380 raft_consensus.cc:686] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [term 1 LEADER]: Becoming Leader. State: Replica: eed19806e0aa49c2b137ec0fe750aa09, State: Running, Role: LEADER
I20260925 22:04:49.748056 31380 consensus_queue.cc:237] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed19806e0aa49c2b137ec0fe750aa09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43615 } }
I20260925 22:04:49.748255 31374 sys_catalog.cc:564] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:49.749336 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:49.743063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42035 (local address 127.9.112.126:43615)
0925 22:04:49.743285 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:49.743294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:49.743326 (+ 32us) server_negotiation.cc:407] Connection header received
0925 22:04:49.744013 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:49.744023 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:49.744081 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:49.744223 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:49.744615 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:49.745601 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:49.746970 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:49.747499 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:49.747595 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:49.748488 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:49.748493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:49.748494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:49.748734 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:49.748781 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:49.748801 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:49.748907 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:49.749070 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 22:04:49.749106 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260925 22:04:49.749506 31381 sys_catalog.cc:455] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eed19806e0aa49c2b137ec0fe750aa09. Latest consensus state: current_term: 1 leader_uuid: "eed19806e0aa49c2b137ec0fe750aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed19806e0aa49c2b137ec0fe750aa09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43615 } } }
I20260925 22:04:49.749511 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eed19806e0aa49c2b137ec0fe750aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed19806e0aa49c2b137ec0fe750aa09" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 43615 } } }
I20260925 22:04:49.749599 31381 sys_catalog.cc:458] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:49.749621 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:49.749984 31393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:49.750569 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:49.750933 31393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:49.753139 31393 catalog_manager.cc:1347] Generated new cluster ID: b6efae4a33db4025885a09a1a37cf7d9
I20260925 22:04:49.753194 31393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:49.756274 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:49.752712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33457 (local address 127.9.112.126:43615)
0925 22:04:49.752761 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:49.752764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:49.752909 (+ 145us) server_negotiation.cc:407] Connection header received
0925 22:04:49.752985 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:49.752985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:49.752998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:49.753051 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:49.753198 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:49.753739 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:49.754940 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:49.755179 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:49.755253 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:49.755992 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:49.755994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:49.755994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:49.755999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:49.756019 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:49.756024 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:49.756075 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:49.756145 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 22:04:49.756156 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 22:04:49.775770 31393 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:49.776657 31393 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:49.781675 31393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eed19806e0aa49c2b137ec0fe750aa09: Generated new TSK 0
I20260925 22:04:49.781941 31393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:49.789615 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31306
W20260925 22:04:49.794728 31291 connection.cc:535] client connection to 127.9.112.126:43615 recv error: Network error: failed to read from TLS socket (remote: 127.9.112.126:43615): Connection reset by peer (error 104)
2026-09-25T22:04:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:49 dist-test-slave-794d krb5kdc[31405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:49 dist-test-slave-794d krb5kdc[31405](info): set up 2 sockets
Sep 25 22:04:49 dist-test-slave-794d krb5kdc[31405](info): commencing operation
krb5kdc: starting...
W20260925 22:04:50.764629 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:50 dist-test-slave-794d krb5kdc[31405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:50.900561 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:41891
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:41891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:51.000268 31418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:51.000591 31418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:51.000631 31418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:51.002101 31418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:51.002167 31418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:51.002187 31418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:51.002203 31418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:51.002218 31418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:51.005059 31418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:41891
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:41891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:51.005582 31418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:51.005995 31418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 22:04:51.012187 31418 server_base.cc:1034] running on GCE node
W20260925 22:04:51.012161 31423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:51.012151 31424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:51.012161 31426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:51.012794 31418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:51.013154 31418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:51.014323 31418 hybrid_clock.cc:648] HybridClock initialized: now 1790373891014280 us; error 64 us; skew 500 ppm
Sep 25 22:04:51 dist-test-slave-794d krb5kdc[31405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373891, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:51.017390 31418 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:51.017967 31418 webserver.cc:460] Webserver started at http://127.9.112.126:33219/ using document root <none> and password file <none>
I20260925 22:04:51.018224 31418 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:51.018280 31418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:51.018428 31418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:51.019696 31418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "0097ba6a8c024f0fae994d6f18d28f87"
format_stamp: "Formatted at 2026-09-25 22:04:51 on dist-test-slave-794d"
server_key: "ec1166ea15d795b9dac3c81ce3147fbd"
server_key_iv: "6e0c03892f7780ce8b95721c4046f9ed"
server_key_version: "encryptionkey@0"
I20260925 22:04:51.020082 31418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "0097ba6a8c024f0fae994d6f18d28f87"
format_stamp: "Formatted at 2026-09-25 22:04:51 on dist-test-slave-794d"
server_key: "ec1166ea15d795b9dac3c81ce3147fbd"
server_key_iv: "6e0c03892f7780ce8b95721c4046f9ed"
server_key_version: "encryptionkey@0"
I20260925 22:04:51.021775 31418 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260925 22:04:51.022728 31433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:51.023030 31418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:51.023133 31418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "0097ba6a8c024f0fae994d6f18d28f87"
format_stamp: "Formatted at 2026-09-25 22:04:51 on dist-test-slave-794d"
server_key: "ec1166ea15d795b9dac3c81ce3147fbd"
server_key_iv: "6e0c03892f7780ce8b95721c4046f9ed"
server_key_version: "encryptionkey@0"
I20260925 22:04:51.023208 31418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:51.027390 31418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:51.029932 31418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:51.030078 31418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:51.034929 31418 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:41891
I20260925 22:04:51.035109 31485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:41891 every 8 connection(s)
I20260925 22:04:51.035355 31418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:51.036557 31486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:51.036753 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31418
I20260925 22:04:51.036864 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:51.037066 9665 external_mini_cluster.cc:1428] Setting key c63b4cc03ffdbf93f0e9e236c93e5597
I20260925 22:04:51.039523 31486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87: Bootstrap starting.
I20260925 22:04:51.040172 31486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:51.040222 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:51.037682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37657 (local address 127.9.112.126:41891)
0925 22:04:51.037918 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:51.037928 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:51.037956 (+ 28us) server_negotiation.cc:407] Connection header received
0925 22:04:51.038503 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:51.038511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:51.038539 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:51.038691 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:51.039603 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:51.039608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:51.039608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:51.039614 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:51.039669 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:51.039681 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:51.039744 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:51.039890 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 22:04:51.040042 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":114,"threads_started":1}
I20260925 22:04:51.040496 31486 log.cc:824] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:51.041424 31486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87: No bootstrap required, opened a new log
I20260925 22:04:51.041466 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:51.044845 31486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0097ba6a8c024f0fae994d6f18d28f87" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41891 } }
I20260925 22:04:51.045079 31486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:51.045087 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:51.043681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49861 (local address 127.9.112.126:41891)
0925 22:04:51.043741 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:51.043743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:51.043890 (+ 147us) server_negotiation.cc:407] Connection header received
0925 22:04:51.043933 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:51.043934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:51.043940 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:51.044011 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:51.044773 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:51.044774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:51.044775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:51.044779 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:51.044796 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:51.044801 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:51.044863 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:51.044984 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 22:04:51.045005 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 22:04:51.045146 31486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0097ba6a8c024f0fae994d6f18d28f87, State: Initialized, Role: FOLLOWER
I20260925 22:04:51.045336 31486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0097ba6a8c024f0fae994d6f18d28f87" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41891 } }
I20260925 22:04:51.045432 31486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:51.045467 31486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:51.045521 31486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:51.046262 31486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0097ba6a8c024f0fae994d6f18d28f87" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41891 } }
I20260925 22:04:51.046484 31486 leader_election.cc:302] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0097ba6a8c024f0fae994d6f18d28f87; no voters:
I20260925 22:04:51.046690 31486 leader_election.cc:288] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:51.046789 31496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:51.046988 31486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:51.047024 31496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [term 1 LEADER]: Becoming Leader. State: Replica: 0097ba6a8c024f0fae994d6f18d28f87, State: Running, Role: LEADER
I20260925 22:04:51.047183 31496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0097ba6a8c024f0fae994d6f18d28f87" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41891 } }
I20260925 22:04:51.047634 31498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0097ba6a8c024f0fae994d6f18d28f87. Latest consensus state: current_term: 1 leader_uuid: "0097ba6a8c024f0fae994d6f18d28f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0097ba6a8c024f0fae994d6f18d28f87" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41891 } } }
I20260925 22:04:51.047739 31498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:51.047988 31505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:51.048156 31497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0097ba6a8c024f0fae994d6f18d28f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0097ba6a8c024f0fae994d6f18d28f87" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 41891 } } }
I20260925 22:04:51.048210 31497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:51.048628 31505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:51.050555 31505 catalog_manager.cc:1347] Generated new cluster ID: d865e931a37443b1a8792049fa1344cb
I20260925 22:04:51.050602 31505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:51.062307 31505 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:51.062525 31505 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:51.067070 31505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0097ba6a8c024f0fae994d6f18d28f87: Generated new TSK 0
I20260925 22:04:51.067312 31505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:51.075564 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31418
2026-09-25T22:04:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:51 dist-test-slave-794d krb5kdc[31517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:51 dist-test-slave-794d krb5kdc[31517](info): set up 2 sockets
Sep 25 22:04:51 dist-test-slave-794d krb5kdc[31517](info): commencing operation
krb5kdc: starting...
W20260925 22:04:52.056926 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:52 dist-test-slave-794d krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:52.196116 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:37161
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:52.290933 31530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:52.291195 31530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:52.291221 31530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:52.292940 31530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:52.293013 31530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:52.293040 31530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:52.293051 31530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:52.293064 31530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:52.294798 31530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:37161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:37161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:52.295091 31530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:52.295325 31530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:52.300849 31538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:52.300848 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:52.300971 31535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:52.301132 31530 server_base.cc:1034] running on GCE node
I20260925 22:04:52.301604 31530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:52.301975 31530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:52.303189 31530 hybrid_clock.cc:648] HybridClock initialized: now 1790373892303169 us; error 41 us; skew 500 ppm
Sep 25 22:04:52 dist-test-slave-794d krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:52.306016 31530 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:52.306509 31530 webserver.cc:460] Webserver started at http://127.9.112.126:34725/ using document root <none> and password file <none>
I20260925 22:04:52.306748 31530 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:52.306800 31530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:52.306936 31530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:52.308070 31530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "1da278eb989d4dbb8579c8e352ed5d14"
format_stamp: "Formatted at 2026-09-25 22:04:52 on dist-test-slave-794d"
server_key: "77447b7ea448104d6188685277e9756f"
server_key_iv: "479d63839eddb0660bb737478e138e62"
server_key_version: "encryptionkey@0"
I20260925 22:04:52.308473 31530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "1da278eb989d4dbb8579c8e352ed5d14"
format_stamp: "Formatted at 2026-09-25 22:04:52 on dist-test-slave-794d"
server_key: "77447b7ea448104d6188685277e9756f"
server_key_iv: "479d63839eddb0660bb737478e138e62"
server_key_version: "encryptionkey@0"
I20260925 22:04:52.309937 31530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:52.310989 31545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:52.311299 31530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:52.311399 31530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "1da278eb989d4dbb8579c8e352ed5d14"
format_stamp: "Formatted at 2026-09-25 22:04:52 on dist-test-slave-794d"
server_key: "77447b7ea448104d6188685277e9756f"
server_key_iv: "479d63839eddb0660bb737478e138e62"
server_key_version: "encryptionkey@0"
I20260925 22:04:52.311474 31530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:52.328394 31530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:52.330564 31530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:52.330703 31530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:52.335142 31530 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:37161
I20260925 22:04:52.335253 31597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:37161 every 8 connection(s)
I20260925 22:04:52.335567 31530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:52.336650 31598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:52.339453 31598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14: Bootstrap starting.
I20260925 22:04:52.340138 31598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:52.340488 31598 log.cc:824] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:52.341512 31598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14: No bootstrap required, opened a new log
I20260925 22:04:52.342702 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31530
I20260925 22:04:52.342813 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:52.343007 9665 external_mini_cluster.cc:1428] Setting key 5d6e51548e623a674ba242785dc35f45
I20260925 22:04:52.343536 31598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da278eb989d4dbb8579c8e352ed5d14" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37161 } }
I20260925 22:04:52.343761 31598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:52.343803 31598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da278eb989d4dbb8579c8e352ed5d14, State: Initialized, Role: FOLLOWER
I20260925 22:04:52.343967 31598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da278eb989d4dbb8579c8e352ed5d14" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37161 } }
I20260925 22:04:52.344043 31598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:52.344065 31598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:52.344095 31598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:52.344986 31598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da278eb989d4dbb8579c8e352ed5d14" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37161 } }
I20260925 22:04:52.345119 31598 leader_election.cc:302] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1da278eb989d4dbb8579c8e352ed5d14; no voters:
I20260925 22:04:52.345335 31598 leader_election.cc:288] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:52.345443 31603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:52.345649 31598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:52.345705 31603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [term 1 LEADER]: Becoming Leader. State: Replica: 1da278eb989d4dbb8579c8e352ed5d14, State: Running, Role: LEADER
I20260925 22:04:52.345881 31603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da278eb989d4dbb8579c8e352ed5d14" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37161 } }
I20260925 22:04:52.346349 31604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1da278eb989d4dbb8579c8e352ed5d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da278eb989d4dbb8579c8e352ed5d14" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37161 } } }
I20260925 22:04:52.346343 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1da278eb989d4dbb8579c8e352ed5d14. Latest consensus state: current_term: 1 leader_uuid: "1da278eb989d4dbb8579c8e352ed5d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da278eb989d4dbb8579c8e352ed5d14" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37161 } } }
I20260925 22:04:52.346451 31604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:52.346454 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:52.346987 31610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:52.347672 31610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:52.349743 31610 catalog_manager.cc:1347] Generated new cluster ID: 387032734486438b9dbdb9a35d902e21
I20260925 22:04:52.349802 31610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 22:04:52 dist-test-slave-794d krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:52.359848 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:52.343667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36997 (local address 127.9.112.126:37161)
0925 22:04:52.343909 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:52.343920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:52.343962 (+ 42us) server_negotiation.cc:407] Connection header received
0925 22:04:52.344134 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:52.344144 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:52.344208 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:52.344364 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:52.345219 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:52.347026 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:52.348234 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:52.348926 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:52.349065 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:52.355199 (+ 6134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:52.355207 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:52.355207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:52.355225 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:52.357559 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:52.358117 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:52.358121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:52.358121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:52.358270 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:52.358493 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:52.358494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:52.358494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:52.358862 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:52.359041 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:52.359197 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 22:04:52.359374 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":94,"threads_started":1}
I20260925 22:04:52.360291 31610 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:52.360853 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:52.361093 31610 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:52.369796 31610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1da278eb989d4dbb8579c8e352ed5d14: Generated new TSK 0
I20260925 22:04:52.370033 31610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:52.370587 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:52.362496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37611 (local address 127.9.112.126:37161)
0925 22:04:52.362545 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:52.362547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:52.362562 (+ 15us) server_negotiation.cc:407] Connection header received
0925 22:04:52.362647 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:52.362648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:52.362661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:52.362738 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:52.363392 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:52.364253 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:52.365360 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:52.365720 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:52.365797 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:52.367294 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:52.367296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:52.367296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:52.367304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:52.369201 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:52.369672 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:52.369673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:52.369674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:52.369753 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:52.369964 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:52.369965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:52.369965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:52.370100 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:52.370184 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:52.370386 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 22:04:52.370498 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:04:52.373049 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31530
2026-09-25T22:04:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:52 dist-test-slave-794d krb5kdc[31629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:52 dist-test-slave-794d krb5kdc[31629](info): set up 2 sockets
Sep 25 22:04:52 dist-test-slave-794d krb5kdc[31629](info): commencing operation
krb5kdc: starting...
W20260925 22:04:53.356648 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:53 dist-test-slave-794d krb5kdc[31629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:53.492501 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38903
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:38903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:53.575364 31642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:53.575599 31642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:53.575634 31642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:53.576869 31642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:53.576921 31642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:53.576941 31642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:53.576956 31642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:53.576972 31642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:53.578536 31642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:38903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:53.578809 31642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:53.579027 31642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:53.584558 31647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:53.584614 31642 server_base.cc:1034] running on GCE node
W20260925 22:04:53.584558 31650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:53.584599 31648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:53.585198 31642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:53.585566 31642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:53.586759 31642 hybrid_clock.cc:648] HybridClock initialized: now 1790373893586732 us; error 44 us; skew 500 ppm
Sep 25 22:04:53 dist-test-slave-794d krb5kdc[31629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373893, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:53.593082 31642 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:53.593612 31642 webserver.cc:460] Webserver started at http://127.9.112.126:37659/ using document root <none> and password file <none>
I20260925 22:04:53.593871 31642 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:53.593924 31642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:53.594065 31642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:53.595292 31642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "45d4aef8ee0847d58625d70728965575"
format_stamp: "Formatted at 2026-09-25 22:04:53 on dist-test-slave-794d"
server_key: "61c308345ac2fdacf02fc6471d180026"
server_key_iv: "126c15365d26db976074e3137856de68"
server_key_version: "encryptionkey@0"
I20260925 22:04:53.595645 31642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "45d4aef8ee0847d58625d70728965575"
format_stamp: "Formatted at 2026-09-25 22:04:53 on dist-test-slave-794d"
server_key: "61c308345ac2fdacf02fc6471d180026"
server_key_iv: "126c15365d26db976074e3137856de68"
server_key_version: "encryptionkey@0"
I20260925 22:04:53.597163 31642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:53.598073 31657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:53.598327 31642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:53.598428 31642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "45d4aef8ee0847d58625d70728965575"
format_stamp: "Formatted at 2026-09-25 22:04:53 on dist-test-slave-794d"
server_key: "61c308345ac2fdacf02fc6471d180026"
server_key_iv: "126c15365d26db976074e3137856de68"
server_key_version: "encryptionkey@0"
I20260925 22:04:53.598505 31642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:53.608173 31642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:53.610535 31642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:53.610661 31642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:53.615288 31642 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:38903
I20260925 22:04:53.615353 31709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:38903 every 8 connection(s)
I20260925 22:04:53.615713 31642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:53.616803 31710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:53.618580 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31642
I20260925 22:04:53.618674 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:53.618886 9665 external_mini_cluster.cc:1428] Setting key 4be9221e70e8d786da05ec6d37322a0c
I20260925 22:04:53.619313 31710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575: Bootstrap starting.
I20260925 22:04:53.620081 31710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:53.620446 31710 log.cc:824] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:53.621394 31710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575: No bootstrap required, opened a new log
I20260925 22:04:53.623536 31710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d4aef8ee0847d58625d70728965575" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38903 } }
I20260925 22:04:53.623718 31710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:53.623755 31710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d4aef8ee0847d58625d70728965575, State: Initialized, Role: FOLLOWER
I20260925 22:04:53.623924 31710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d4aef8ee0847d58625d70728965575" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38903 } }
I20260925 22:04:53.624011 31710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:53.624042 31710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:53.624089 31710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:53.624837 31710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d4aef8ee0847d58625d70728965575" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38903 } }
I20260925 22:04:53.624960 31710 leader_election.cc:302] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45d4aef8ee0847d58625d70728965575; no voters:
I20260925 22:04:53.625140 31710 leader_election.cc:288] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:53.625202 31715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:53.625389 31715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [term 1 LEADER]: Becoming Leader. State: Replica: 45d4aef8ee0847d58625d70728965575, State: Running, Role: LEADER
I20260925 22:04:53.625476 31710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:53.625552 31713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:53.619496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32893 (local address 127.9.112.126:38903)
0925 22:04:53.619743 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:53.619755 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:53.619793 (+ 38us) server_negotiation.cc:407] Connection header received
0925 22:04:53.620409 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:53.620418 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:53.620479 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:53.620672 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:53.620914 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:53.622286 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:53.623147 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:53.623769 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:53.623882 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:53.624681 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:53.624685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:53.624686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:53.624907 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:53.624950 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:53.624965 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:53.625037 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:53.625117 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 22:04:53.625314 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":113,"threads_started":1}
I20260925 22:04:53.625556 31715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d4aef8ee0847d58625d70728965575" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38903 } }
I20260925 22:04:53.626092 31716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45d4aef8ee0847d58625d70728965575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d4aef8ee0847d58625d70728965575" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38903 } } }
I20260925 22:04:53.626482 31716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:53.626549 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:53.626405 31717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45d4aef8ee0847d58625d70728965575. Latest consensus state: current_term: 1 leader_uuid: "45d4aef8ee0847d58625d70728965575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d4aef8ee0847d58625d70728965575" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 38903 } } }
I20260925 22:04:53.626919 31717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575 [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:53.627070 31724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:53.627650 31724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:53.629400 31724 catalog_manager.cc:1347] Generated new cluster ID: 72070308467c49158d60ba79a434d916
I20260925 22:04:53.629452 31724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:53.632494 31713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:53.628805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40555 (local address 127.9.112.126:38903)
0925 22:04:53.628848 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:53.628850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:53.628861 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:53.628962 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:53.628963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:53.628975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:53.629047 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:53.629230 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:53.629971 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:53.631073 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:53.631369 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:53.631411 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:53.632205 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:53.632206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:53.632207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:53.632212 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:53.632226 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:53.632230 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:53.632256 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:53.632383 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 22:04:53.632409 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:04:53.645457 31724 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:53.646050 31724 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:53.653033 31724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45d4aef8ee0847d58625d70728965575: Generated new TSK 0
I20260925 22:04:53.653237 31724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:53.665072 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31642
2026-09-25T22:04:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:53 dist-test-slave-794d krb5kdc[31741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:53 dist-test-slave-794d krb5kdc[31741](info): set up 2 sockets
Sep 25 22:04:53 dist-test-slave-794d krb5kdc[31741](info): commencing operation
krb5kdc: starting...
W20260925 22:04:54.628391 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:54 dist-test-slave-794d krb5kdc[31741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:54.765076 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:37321
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:37321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:54.854564 31754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:54.854797 31754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:54.854821 31754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:54.856015 31754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:54.856066 31754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:54.856079 31754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:54.856091 31754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:54.856101 31754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:54.857770 31754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:37321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:37321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:54.858085 31754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:54.858336 31754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:54.864959 31762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:54.864962 31760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:54.864961 31759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:54.865206 31754 server_base.cc:1034] running on GCE node
I20260925 22:04:54.865540 31754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:54.865880 31754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:54.867033 31754 hybrid_clock.cc:648] HybridClock initialized: now 1790373894867010 us; error 52 us; skew 500 ppm
Sep 25 22:04:54 dist-test-slave-794d krb5kdc[31741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:54.869920 31754 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:54.870440 31754 webserver.cc:460] Webserver started at http://127.9.112.126:35113/ using document root <none> and password file <none>
I20260925 22:04:54.870678 31754 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:54.870728 31754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:54.870879 31754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:54.872049 31754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "6069a7f215e44dffb7fadbc54853736a"
format_stamp: "Formatted at 2026-09-25 22:04:54 on dist-test-slave-794d"
server_key: "61fec12aa17af1b9c3833ba84f061ab3"
server_key_iv: "e601f3cba195a45dcc264fde0c8a1c2d"
server_key_version: "encryptionkey@0"
I20260925 22:04:54.872452 31754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "6069a7f215e44dffb7fadbc54853736a"
format_stamp: "Formatted at 2026-09-25 22:04:54 on dist-test-slave-794d"
server_key: "61fec12aa17af1b9c3833ba84f061ab3"
server_key_iv: "e601f3cba195a45dcc264fde0c8a1c2d"
server_key_version: "encryptionkey@0"
I20260925 22:04:54.874177 31754 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260925 22:04:54.874998 31769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:54.875205 31754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:54.875294 31754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "6069a7f215e44dffb7fadbc54853736a"
format_stamp: "Formatted at 2026-09-25 22:04:54 on dist-test-slave-794d"
server_key: "61fec12aa17af1b9c3833ba84f061ab3"
server_key_iv: "e601f3cba195a45dcc264fde0c8a1c2d"
server_key_version: "encryptionkey@0"
I20260925 22:04:54.875399 31754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:54.879252 31754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:54.881656 31754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:54.881811 31754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:54.886739 31754 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:37321
I20260925 22:04:54.886806 31821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:37321 every 8 connection(s)
I20260925 22:04:54.887277 31754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:54.888834 31822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:54.891501 31822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a: Bootstrap starting.
I20260925 22:04:54.891975 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31754
I20260925 22:04:54.892107 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:54.892256 31822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:54.892313 9665 external_mini_cluster.cc:1428] Setting key 4bd4eb008b50db93e9a91182652c3099
I20260925 22:04:54.892632 31822 log.cc:824] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:54.893585 31822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a: No bootstrap required, opened a new log
I20260925 22:04:54.895550 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:54.893062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37867 (local address 127.9.112.126:37321)
0925 22:04:54.893295 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:54.893306 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:54.893347 (+ 41us) server_negotiation.cc:407] Connection header received
0925 22:04:54.893983 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:54.893992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:54.894025 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:54.894196 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:54.894986 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:54.894990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:54.894990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:54.894995 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:54.895045 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:54.895058 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:54.895139 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:54.895226 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 22:04:54.895311 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260925 22:04:54.896441 31822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6069a7f215e44dffb7fadbc54853736a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37321 } }
I20260925 22:04:54.896651 31822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 22:04:54.896701 31822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6069a7f215e44dffb7fadbc54853736a, State: Initialized, Role: FOLLOWER
I20260925 22:04:54.896835 31822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6069a7f215e44dffb7fadbc54853736a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37321 } }
I20260925 22:04:54.896948 31822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 22:04:54.896986 31822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 22:04:54.897047 31822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:54.897116 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:54.897944 31822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6069a7f215e44dffb7fadbc54853736a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37321 } }
I20260925 22:04:54.898083 31822 leader_election.cc:302] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6069a7f215e44dffb7fadbc54853736a; no voters:
I20260925 22:04:54.898284 31822 leader_election.cc:288] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 22:04:54.898343 31828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:54.898584 31822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:54.898608 31828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [term 1 LEADER]: Becoming Leader. State: Replica: 6069a7f215e44dffb7fadbc54853736a, State: Running, Role: LEADER
I20260925 22:04:54.898821 31828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6069a7f215e44dffb7fadbc54853736a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37321 } }
I20260925 22:04:54.899382 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6069a7f215e44dffb7fadbc54853736a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6069a7f215e44dffb7fadbc54853736a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37321 } } }
I20260925 22:04:54.899493 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:54.899726 31833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6069a7f215e44dffb7fadbc54853736a. Latest consensus state: current_term: 1 leader_uuid: "6069a7f215e44dffb7fadbc54853736a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6069a7f215e44dffb7fadbc54853736a" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 37321 } } }
I20260925 22:04:54.899796 31833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:54.899780 31841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:54.901021 31841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:54.903083 31841 catalog_manager.cc:1347] Generated new cluster ID: 0a8a29bbf798482eba517a4c16da05f0
I20260925 22:04:54.903155 31841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:54.905135 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:54.898848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53365 (local address 127.9.112.126:37321)
0925 22:04:54.898888 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:54.898890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:54.899279 (+ 389us) server_negotiation.cc:407] Connection header received
0925 22:04:54.899878 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:54.899880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:54.899886 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:54.899989 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:54.904854 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:54.904856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:54.904856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 22:04:54.904862 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:54.904886 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 22:04:54.904891 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:54.904963 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:54.905041 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 22:04:54.905058 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 22:04:54.930282 31841 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:54.930544 31841 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:54.938341 31841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6069a7f215e44dffb7fadbc54853736a: Generated new TSK 0
I20260925 22:04:54.938562 31841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:54.945206 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31754
2026-09-25T22:04:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1281 ms)
[----------] 18 tests from AuthTokenIssuingTest (23231 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:55 dist-test-slave-794d krb5kdc[31853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:55 dist-test-slave-794d krb5kdc[31853](info): set up 2 sockets
Sep 25 22:04:55 dist-test-slave-794d krb5kdc[31853](info): commencing operation
krb5kdc: starting...
W20260925 22:04:55.936967 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:56.080055 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40269
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40269,127.9.112.125:44907,127.9.112.124:41263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:56.173990 31866 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:04:56.174278 31866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:56.174306 31866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:56.174336 31866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:56.175899 31866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:56.175967 31866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:56.175987 31866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:56.176002 31866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:56.176026 31866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:56.177847 31866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40269,127.9.112.125:44907,127.9.112.124:41263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:56.178140 31866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:56.178369 31866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:56.183804 31871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:56.183837 31874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:56.183802 31872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:56.183825 31866 server_base.cc:1034] running on GCE node
I20260925 22:04:56.184437 31866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:56.184790 31866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:56.185983 31866 hybrid_clock.cc:648] HybridClock initialized: now 1790373896185952 us; error 41 us; skew 500 ppm
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:56.188946 31866 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:56.189476 31866 webserver.cc:460] Webserver started at http://127.9.112.126:37303/ using document root <none> and password file <none>
I20260925 22:04:56.189738 31866 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:56.189790 31866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:56.189908 31866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:56.191013 31866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "4c7ca7ddfff141f1addfd3902ea0a52b"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "551afac949ae1326b9b5ac9aa73097ed"
server_key_iv: "a157dc21a36e1f2d920a67e4b73ca297"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.191502 31866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "4c7ca7ddfff141f1addfd3902ea0a52b"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "551afac949ae1326b9b5ac9aa73097ed"
server_key_iv: "a157dc21a36e1f2d920a67e4b73ca297"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.193238 31866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 22:04:56.194308 31881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:56.194597 31866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:56.194710 31866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "4c7ca7ddfff141f1addfd3902ea0a52b"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "551afac949ae1326b9b5ac9aa73097ed"
server_key_iv: "a157dc21a36e1f2d920a67e4b73ca297"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.194788 31866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:56.210723 31866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:56.213372 31866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:56.213578 31866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:56.219079 31866 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40269
I20260925 22:04:56.219201 31933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40269 every 8 connection(s)
I20260925 22:04:56.219571 31866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:56.221438 31934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:56.223917 31934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:56.226697 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31866
I20260925 22:04:56.226814 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:56.227018 9665 external_mini_cluster.cc:1428] Setting key 7f30d0e36384390c939f86b08d1abdc7
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.125@KRBTEST.COM; defaulting to no policy
I20260925 22:04:56.240805 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.225070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.9.112.126:40269)
0925 22:04:56.225347 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.225358 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.225591 (+ 233us) server_negotiation.cc:407] Connection header received
0925 22:04:56.226455 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.226457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.226481 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.226591 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.227346 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.228475 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.229464 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.229816 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.229928 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.234412 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.234423 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.234423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.234433 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.236707 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.237146 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.237199 (+ 53us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.237200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.237320 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.237536 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.237537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.237537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.237827 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.237911 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.238100 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.240264 (+ 2164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260925 22:04:56.240803 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.224930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40269 (local address 127.0.0.1:50368)
0925 22:04:56.225331 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.225393 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.226113 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.226606 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.226609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.227018 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.227245 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.227250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.228588 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.228589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.229378 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.229381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.229925 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.229926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.229994 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.230736 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.230749 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.234204 (+ 3455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.236825 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.236832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.236836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.237062 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.237382 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.237383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.237383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.237479 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.237891 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.237897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.237966 (+ 69us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.238055 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.240398 (+ 2343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":207,"thread_start_us":175,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.125@KRBTEST.COM" created.
I20260925 22:04:56.243069 31934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:56.243700 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.243475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:57990)
0925 22:04:56.243582 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.243660 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260925 22:04:56.243971 31885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
I20260925 22:04:56.244746 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.244573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58002)
0925 22:04:56.244673 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.244715 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260925 22:04:56.244971 31934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:44907: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab.
Entry for principal kudu/127.9.112.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab.
I20260925 22:04:56.272055 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:44907
--webserver_interface=127.9.112.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab
--principal=kudu/127.9.112.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40269,127.9.112.125:44907,127.9.112.124:41263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:04:56.295403 31934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } attempt: 1
I20260925 22:04:56.296155 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.295873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58006)
0925 22:04:56.296006 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.296088 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 22:04:56.296805 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.296684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58014)
0925 22:04:56.296746 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.296782 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 22:04:56.296913 31934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:44907: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
I20260925 22:04:56.365080 31934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } attempt: 2
I20260925 22:04:56.365783 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.365568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58020)
0925 22:04:56.365659 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.365734 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 22:04:56.366421 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.366192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58028)
0925 22:04:56.366324 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.366376 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260925 22:04:56.366595 31934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:44907: Network error: Client connection negotiation failed: client connection to 127.9.112.125:44907: connect: Connection refused (error 111)
W20260925 22:04:56.375630 31941 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:04:56.375913 31941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:56.375941 31941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:56.375970 31941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:56.377347 31941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:56.377408 31941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:56.377429 31941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:56.377445 31941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:56.377461 31941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:56.379552 31941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40269,127.9.112.125:44907,127.9.112.124:41263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab
--principal=kudu/127.9.112.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:44907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.112.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:56.379901 31941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:56.380163 31941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:56.386261 31949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:56.386261 31946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:56.386261 31947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:56.386934 31941 server_base.cc:1034] running on GCE node
I20260925 22:04:56.387259 31941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:56.387797 31941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:56.388998 31941 hybrid_clock.cc:648] HybridClock initialized: now 1790373896388969 us; error 58 us; skew 500 ppm
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:56.392719 31941 init.cc:377] Logged in from keytab as kudu/127.9.112.125@KRBTEST.COM (short username kudu)
I20260925 22:04:56.393373 31941 webserver.cc:460] Webserver started at http://127.9.112.125:37667/ using document root <none> and password file <none>
I20260925 22:04:56.393644 31941 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:56.393700 31941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:56.393844 31941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:56.395135 31941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data/instance:
uuid: "da7c6a0bce0a416c8c619e033e357fe9"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "2139181ee918e514e3f68c2e4c5703e3"
server_key_iv: "65b33c4da0ed6e7f70fccf6413b46935"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.395526 31941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal/instance:
uuid: "da7c6a0bce0a416c8c619e033e357fe9"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "2139181ee918e514e3f68c2e4c5703e3"
server_key_iv: "65b33c4da0ed6e7f70fccf6413b46935"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.397579 31941 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 22:04:56.398722 31956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:56.398965 31941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:56.399067 31941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal
uuid: "da7c6a0bce0a416c8c619e033e357fe9"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "2139181ee918e514e3f68c2e4c5703e3"
server_key_iv: "65b33c4da0ed6e7f70fccf6413b46935"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.399139 31941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:56.416023 31941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:56.418300 31941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:56.418421 31941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:56.422855 31941 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.125:44907
I20260925 22:04:56.422976 32008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.125:44907 every 8 connection(s)
I20260925 22:04:56.423296 31941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
I20260925 22:04:56.424464 32009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:56.426378 32009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:56.430488 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 31941
I20260925 22:04:56.430599 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-1/wal/instance
I20260925 22:04:56.430783 9665 external_mini_cluster.cc:1428] Setting key 0b133234c332cf3ec9dca604667d29c9
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:56.436483 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.427258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.9.112.126:40269)
0925 22:04:56.427377 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.427380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.427631 (+ 251us) server_negotiation.cc:407] Connection header received
0925 22:04:56.428222 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.428224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.428237 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.428295 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.429080 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.429621 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.430687 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.430922 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.430980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.433568 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.433569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.433570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.433579 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.435124 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.435515 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.435516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.435516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.435569 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.435787 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.435788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.435788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.435908 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.435965 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.436262 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.436379 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 22:04:56.436673 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.427212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40269 (local address 127.0.0.1:50374)
0925 22:04:56.427467 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.427512 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.428112 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.428398 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.428402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.428748 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.428984 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.428989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.429717 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.429718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.430603 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.430606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.431015 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.431016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.431151 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.431769 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.431780 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.433479 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.435205 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.435213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.435216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.435448 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.435622 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.435623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.435623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.435732 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.435962 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.435968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.436113 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.436221 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.436424 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":110,"threads_started":1}
I20260925 22:04:56.437238 32009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.112.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.124@KRBTEST.COM" created.
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:04:56.445895 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.437547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58036)
0925 22:04:56.437648 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.437666 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.437747 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.437923 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.437924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.438106 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.438198 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.438200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.439203 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.439204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.439833 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.439835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.440371 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.440372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.440445 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.440907 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.440912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.442115 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.444525 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.444526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.444527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.444751 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.445087 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.445088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.445088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.445156 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.445471 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.445472 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.445542 (+ 70us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.445633 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.445772 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 22:04:56.445986 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.437588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.9.112.125:44907)
0925 22:04:56.437763 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.437767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.437784 (+ 17us) server_negotiation.cc:407] Connection header received
0925 22:04:56.437805 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.437807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.437819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.437887 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.438315 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.439098 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.439914 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.440266 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.440332 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.442207 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.442218 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.442218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.442236 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.444427 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.444942 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.444945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.444945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.445013 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.445220 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.445221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.445221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.445405 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.445473 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.445681 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.445855 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260925 22:04:56.447125 32009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:56.447616 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.447435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40888)
0925 22:04:56.447506 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.447572 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 22:04:56.447817 31959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
I20260925 22:04:56.448289 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.448172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40890)
0925 22:04:56.448229 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.448265 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 22:04:56.448485 32009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:41263: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
I20260925 22:04:56.457768 31934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab.
Entry for principal kudu/127.9.112.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab.
I20260925 22:04:56.460095 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:41263
--webserver_interface=127.9.112.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab
--principal=kudu/127.9.112.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40269,127.9.112.125:44907,127.9.112.124:41263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:04:56.466643 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.458377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.9.112.125:44907)
0925 22:04:56.458438 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.458440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.458453 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:04:56.458486 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.458487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.458500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.458559 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.459187 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.459871 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.460968 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.461288 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.461369 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.463439 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.463441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.463441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.463451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.465244 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.465736 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.465737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.465737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.465827 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.466054 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.466054 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.466054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.466184 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.466241 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.466464 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.466560 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:04:56.466652 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.458247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58050)
0925 22:04:56.458338 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.458352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.458421 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.458590 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.458591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.458750 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.459100 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.459102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.459985 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.459986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.460861 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.460865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.461387 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.461388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.461553 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.462087 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.462091 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.463328 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.465348 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.465349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.465350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.465591 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.465910 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.465911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.465911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.465983 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.466250 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.466251 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.466319 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.466411 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.466536 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 22:04:56.467520 31934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:56.467971 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.467816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40906)
0925 22:04:56.467879 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.467927 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 22:04:56.468461 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.468366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40908)
0925 22:04:56.468401 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.468437 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260925 22:04:56.468633 31934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:41263: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
I20260925 22:04:56.497745 32009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } attempt: 1
I20260925 22:04:56.498425 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.498195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40918)
0925 22:04:56.498314 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.498388 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 22:04:56.498844 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.498734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40920)
0925 22:04:56.498790 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.498823 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 22:04:56.498955 32009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:41263: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
I20260925 22:04:56.499820 31934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } attempt: 1
I20260925 22:04:56.500303 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.500126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40936)
0925 22:04:56.500209 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.500260 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 22:04:56.500713 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.500592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40946)
0925 22:04:56.500655 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.500689 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 22:04:56.500869 31934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:41263: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
W20260925 22:04:56.551630 32016 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:04:56.551889 32016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:56.551908 32016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:56.551934 32016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:56.553363 32016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:56.553440 32016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:56.553455 32016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:56.553467 32016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:56.553478 32016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:56.555238 32016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40269,127.9.112.125:44907,127.9.112.124:41263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab
--principal=kudu/127.9.112.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:41263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.112.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:56.555544 32016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:56.555784 32016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:56.561322 32022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:56.561314 32024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:56.561609 32016 server_base.cc:1034] running on GCE node
W20260925 22:04:56.561314 32021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:56.562201 32016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:56.562604 32016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:56.563799 32016 hybrid_clock.cc:648] HybridClock initialized: now 1790373896563765 us; error 47 us; skew 500 ppm
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:56.566826 32016 init.cc:377] Logged in from keytab as kudu/127.9.112.124@KRBTEST.COM (short username kudu)
I20260925 22:04:56.567109 32009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } attempt: 2
I20260925 22:04:56.567390 32016 webserver.cc:460] Webserver started at http://127.9.112.124:41061/ using document root <none> and password file <none>
I20260925 22:04:56.567603 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.567439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40950)
0925 22:04:56.567515 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.567564 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:04:56.567657 32016 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:56.567732 32016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:56.567873 32016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:56.568042 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.567942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40956)
0925 22:04:56.567991 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.568022 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 22:04:56.568147 32009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:41263: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
I20260925 22:04:56.569129 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data/instance:
uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "f9537180e94b9f2142a687175529d64c"
server_key_iv: "54ea2bb08416b16b9712aaa783bb1de1"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.569444 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal/instance:
uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "f9537180e94b9f2142a687175529d64c"
server_key_iv: "54ea2bb08416b16b9712aaa783bb1de1"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.571004 32016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:04:56.572145 32031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:56.572397 32016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:56.572512 32016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal
uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78"
format_stamp: "Formatted at 2026-09-25 22:04:56 on dist-test-slave-794d"
server_key: "f9537180e94b9f2142a687175529d64c"
server_key_iv: "54ea2bb08416b16b9712aaa783bb1de1"
server_key_version: "encryptionkey@0"
I20260925 22:04:56.572598 32016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:56.576052 31934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } attempt: 2
I20260925 22:04:56.576637 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.576456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40966)
0925 22:04:56.576530 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.576593 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:04:56.577051 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.576949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:40978)
0925 22:04:56.576998 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.577025 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 22:04:56.577135 31934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:41263: Network error: Client connection negotiation failed: client connection to 127.9.112.124:41263: connect: Connection refused (error 111)
I20260925 22:04:56.588101 32016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:56.590476 32016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:56.590602 32016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:56.595218 32016 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.124:41263
I20260925 22:04:56.595340 32083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.124:41263 every 8 connection(s)
I20260925 22:04:56.595676 32016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
I20260925 22:04:56.596942 32084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:56.597069 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 32016
I20260925 22:04:56.597164 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790373628768857-9665-0/minicluster-data/master-2/wal/instance
I20260925 22:04:56.597404 9665 external_mini_cluster.cc:1428] Setting key d3795baac361b50b688cad3d7f03fc66
I20260925 22:04:56.599814 32084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:56.609851 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.598123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.9.112.124:41263)
0925 22:04:56.598353 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.598364 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.598419 (+ 55us) server_negotiation.cc:407] Connection header received
0925 22:04:56.598982 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.598992 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.599085 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.599294 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.599952 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.601253 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.602307 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.603142 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.603253 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.606111 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.606120 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.606121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.606130 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.608367 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.608743 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.608746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.608747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.608830 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.609009 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.609010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.609010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.609267 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.609400 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.609509 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.609657 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":100,"threads_started":1}
I20260925 22:04:56.610018 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.600776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40269 (local address 127.0.0.1:50382)
0925 22:04:56.601018 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.601063 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.601181 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.601379 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.601381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.601774 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.601875 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.601878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.602839 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.602840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.604030 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.604034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.604546 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.604547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.604631 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.605231 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.605244 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.606886 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.608760 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.608772 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.608775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.609016 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.609199 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.609200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.609200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.609291 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.609526 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.609533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.609593 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.609680 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.609838 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":152,"threads_started":1}
I20260925 22:04:56.610081 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.600840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.9.112.126:40269)
0925 22:04:56.600923 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.600925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.601158 (+ 233us) server_negotiation.cc:407] Connection header received
0925 22:04:56.601271 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.601272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.601285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.601341 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.601994 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.602706 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.604142 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.604424 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.604493 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.606983 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.606984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.606984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.606992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.608669 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.609088 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.609090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.609090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.609146 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.609348 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.609349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.609350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.609471 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.609528 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.609908 (+ 380us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.609993 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 22:04:56.610800 32084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:56.618822 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.611108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:44907 (local address 127.0.0.1:58052)
0925 22:04:56.611164 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.611173 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.611204 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.611479 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.611480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.611619 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.611663 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.611664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.612681 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.612683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.613243 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.613245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.613746 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.613748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.613816 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.614250 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.614256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.615808 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.617510 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.617511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.617512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.617732 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.618042 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.618042 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.618043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.618114 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.618404 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.618405 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.618463 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.618547 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.618717 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 22:04:56.618920 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.611272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.9.112.125:44907)
0925 22:04:56.611332 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.611334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.611342 (+ 8us) server_negotiation.cc:407] Connection header received
0925 22:04:56.611369 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.611370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.611381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.611438 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.611739 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.612537 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.613331 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.613633 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.613700 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.615912 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.615914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.615914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.615923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.617391 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.617886 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.617887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.617887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.617963 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.618189 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.618189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.618190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.618333 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.618391 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.618625 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.618720 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:04:56.619578 32084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:56.619676 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.610698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50384 (local address 127.9.112.126:40269)
0925 22:04:56.610754 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.610756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.611108 (+ 352us) server_negotiation.cc:407] Connection header received
0925 22:04:56.611141 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.611142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.611155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.611222 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.611958 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.612717 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.613525 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.613737 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.613817 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.616650 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.616652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.616652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.616660 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.618239 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.618647 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.618649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.618650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.618732 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.619060 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.619062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.619063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.619225 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.619448 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.619487 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.619565 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:04:56.627478 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.619910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:41004)
0925 22:04:56.620079 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.620092 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.620149 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.620282 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.620283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.620483 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.620528 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.620530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.621400 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.621400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.621911 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.621912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.622394 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.622395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.622462 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.623054 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.623060 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.624502 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.626017 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.626018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.626019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.626172 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.626497 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.626499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.626499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.626612 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.627007 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.627008 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.627072 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.627173 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.627297 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260925 22:04:56.627490 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.620007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.9.112.124:41263)
0925 22:04:56.620049 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.620051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.620198 (+ 147us) server_negotiation.cc:407] Connection header received
0925 22:04:56.620219 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.620219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.620229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.620275 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.620650 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.621299 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.621994 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.622315 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.622383 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.624582 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.624583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.624584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.624591 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.625931 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.626236 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.626237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.626237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.626354 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.626779 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.626780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.626780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.626913 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.626980 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.627227 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.627349 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":19}
I20260925 22:04:56.628321 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.620437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.9.112.125:44907)
0925 22:04:56.620520 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.620522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.620558 (+ 36us) server_negotiation.cc:407] Connection header received
0925 22:04:56.620735 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.620736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.620790 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.620862 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.621331 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.622022 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.622847 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.623088 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.623150 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.625547 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.625550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.625550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.625559 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.627169 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.627567 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.627568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.627569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.627624 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.627795 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.627795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.627796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.627926 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.628101 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.628187 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.628254 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 22:04:56.630074 32084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: Bootstrap starting.
I20260925 22:04:56.630241 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:56.631079 32084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:56.631392 32084 log.cc:824] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:56.631233 31908 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50384
I20260925 22:04:56.632534 32084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: No bootstrap required, opened a new log
I20260925 22:04:56.634281 32084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.634440 32084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0cc2abd30b45c6bf1c845cc54d1f78, State: Initialized, Role: FOLLOWER
I20260925 22:04:56.634516 32084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.634953 32084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:56.635028 32090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.635213 32090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:04:56.637048 32101 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:56.637121 32101 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:56.637596 32016 master_runner.cc:186] Error getting master registration for 127.9.112.126:40269: OK, instance_id { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" instance_seqno: 1790373896217550 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:04:56.642139 32016 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:04:56.659330 32009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } attempt: 3
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
I20260925 22:04:56.668330 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.659886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.9.112.124:41263)
0925 22:04:56.659953 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.659956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.659966 (+ 10us) server_negotiation.cc:407] Connection header received
0925 22:04:56.660002 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.660003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.660012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.660059 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.660436 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.660929 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.661884 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.662092 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.662158 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.664775 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.664777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.664777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.664787 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.666551 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.667055 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.667056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.667056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.667130 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.667546 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.667560 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.667560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.667810 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.667902 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.668111 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.668222 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 22:04:56.668396 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.659791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:41008)
0925 22:04:56.659846 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.659860 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.659913 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.660070 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.660071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.660270 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.660324 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.660326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.661003 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.661004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.661807 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.661809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.662163 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.662164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.662215 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.662832 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.662837 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.664632 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.666681 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.666684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.666685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.666963 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.667324 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.667326 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.667326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.667430 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.667904 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.667905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.667985 (+ 80us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.668072 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.668226 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 22:04:56.670426 32009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: Bootstrap starting.
I20260925 22:04:56.671294 32009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:56.671638 32009 log.cc:824] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:56.672708 32009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: No bootstrap required, opened a new log
I20260925 22:04:56.674867 32009 raft_consensus.cc:348] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.675050 32009 raft_consensus.cc:729] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7c6a0bce0a416c8c619e033e357fe9, State: Initialized, Role: FOLLOWER
I20260925 22:04:56.675164 32009 consensus_queue.cc:260] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.675594 32009 sys_catalog.cc:564] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:56.675521 32104 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.675645 32104 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:56.676276 31934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } attempt: 3
W20260925 22:04:56.677740 32115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:56.677816 32115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:56.677659 31941 master_runner.cc:186] Error getting master registration for 127.9.112.126:40269: OK, instance_id { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" instance_seqno: 1790373896217550 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:04:56.680783 31941 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 25 22:04:56 dist-test-slave-794d krb5kdc[31853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
I20260925 22:04:56.685310 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.676797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:41263 (local address 127.0.0.1:41024)
0925 22:04:56.676855 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:56.676869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:56.676922 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:56.677209 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:56.677210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:56.677431 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:56.677485 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.677488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.678470 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.678471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.679240 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:56.679243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.679756 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.679757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:56.679825 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.680494 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:56.680500 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:56.682258 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:56.684235 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.684236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.684237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.684497 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.684675 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:56.684676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:56.684676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:56.684736 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:56.684996 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:56.684997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:56.685045 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 22:04:56.685104 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 22:04:56.685208 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 22:04:56.685310 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.676873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.9.112.124:41263)
0925 22:04:56.676932 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.676934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.676950 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:04:56.676983 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.676984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.676997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.677112 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.677585 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.678272 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.679345 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.679626 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.679682 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.682374 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.682376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.682377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.682386 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.684151 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.684562 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.684563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.684563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.684617 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.684789 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.684790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.684806 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.684940 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.684993 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.685135 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.685208 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 22:04:56.686829 31934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b: Bootstrap starting.
I20260925 22:04:56.687448 31934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:56.687737 31934 log.cc:824] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:56.688596 31934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b: No bootstrap required, opened a new log
I20260925 22:04:56.690006 31934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.690160 31934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7ca7ddfff141f1addfd3902ea0a52b, State: Initialized, Role: FOLLOWER
I20260925 22:04:56.690243 31934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.690593 31934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:56.690604 31908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 22:04:56.690696 31908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:56.691586 31908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.691859 31908 leader_election.cc:288] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [CANDIDATE]: Term 1 election: Requested vote from peers da7c6a0bce0a416c8c619e033e357fe9 (127.9.112.125:44907), 8e0cc2abd30b45c6bf1c845cc54d1f78 (127.9.112.124:41263)
I20260925 22:04:56.692543 31983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da7c6a0bce0a416c8c619e033e357fe9"
I20260925 22:04:56.692701 31983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:56.693252 32058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78"
I20260925 22:04:56.693389 32058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:56.693699 31983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7ca7ddfff141f1addfd3902ea0a52b in term 1.
I20260925 22:04:56.694380 32058 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7ca7ddfff141f1addfd3902ea0a52b in term 1.
I20260925 22:04:56.694758 31884 leader_election.cc:302] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7ca7ddfff141f1addfd3902ea0a52b, 8e0cc2abd30b45c6bf1c845cc54d1f78; no voters:
I20260925 22:04:56.690554 32118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.695187 32118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:56.695287 32118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:56.695374 32118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [term 1 LEADER]: Becoming Leader. State: Replica: 4c7ca7ddfff141f1addfd3902ea0a52b, State: Running, Role: LEADER
I20260925 22:04:56.695513 32118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } }
I20260925 22:04:56.696152 32118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7ca7ddfff141f1addfd3902ea0a52b. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.696273 32118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:56.696898 31866 master_runner.cc:263] No leader master found from master 4c7ca7ddfff141f1addfd3902ea0a52b
I20260925 22:04:56.696956 31866 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:04:56.697080 32136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:56.697558 32136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:56.701627 32058 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [term 1 FOLLOWER]: Refusing update from remote peer 4c7ca7ddfff141f1addfd3902ea0a52b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:56.702121 32134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:56.703660 32090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7ca7ddfff141f1addfd3902ea0a52b. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.703770 32090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:56.704270 31983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [term 1 FOLLOWER]: Refusing update from remote peer 4c7ca7ddfff141f1addfd3902ea0a52b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:56.704593 32134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:56.705922 32104 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7ca7ddfff141f1addfd3902ea0a52b. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.706022 32104 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:56.708029 32134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.708135 32134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:56.708678 32104 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.708773 32104 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:56.708969 32138 mvcc.cc:205] Tried to move back new op lower bound from 7333371480886104064 to 7333371480866283520. Current Snapshot: MvccSnapshot[applied={T|T < 7333371480886104064}]
I20260925 22:04:56.709172 32144 mvcc.cc:205] Tried to move back new op lower bound from 7333371480886104064 to 7333371480866283520. Current Snapshot: MvccSnapshot[applied={T|T < 7333371480886104064}]
I20260925 22:04:56.709515 32118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.709599 32118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:56.709934 32090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c7ca7ddfff141f1addfd3902ea0a52b" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40269 } } peers { permanent_uuid: "da7c6a0bce0a416c8c619e033e357fe9" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 44907 } } peers { permanent_uuid: "8e0cc2abd30b45c6bf1c845cc54d1f78" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 41263 } } }
I20260925 22:04:56.710031 32090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:56.710685 32142 mvcc.cc:205] Tried to move back new op lower bound from 7333371480886104064 to 7333371480866283520. Current Snapshot: MvccSnapshot[applied={T|T < 7333371480886104064}]
I20260925 22:04:56.711580 32136 catalog_manager.cc:1347] Generated new cluster ID: 075efbd0bcbb404a94f99fb00922c28e
I20260925 22:04:56.711633 32136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:56.726081 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.700744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.9.112.124:41263)
0925 22:04:56.700816 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.700819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.712883 (+ 12064us) server_negotiation.cc:407] Connection header received
0925 22:04:56.713957 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.713958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.713971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.714001 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.714704 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.715418 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.716807 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.717088 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.717145 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.721981 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.721983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.721984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.721996 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.723776 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.724205 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.724206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.724207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.724266 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.725419 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.725421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.725421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.725580 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.725666 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.725875 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.725966 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:04:56.731186 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.701033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.9.112.126:40269)
0925 22:04:56.701085 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.701087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.712699 (+ 11612us) server_negotiation.cc:407] Connection header received
0925 22:04:56.712938 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.712940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.712955 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.712986 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.716475 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.717539 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.725730 (+ 8191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.726043 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.726087 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.727860 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.727862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.727862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.727871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.729853 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.730318 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.730319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.730320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.730379 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.730558 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.730559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.730559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.730695 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.730773 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.730980 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.731069 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":260,"server-negotiator.queue_time_us":20}
I20260925 22:04:56.739933 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:56.701446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.9.112.125:44907)
0925 22:04:56.701841 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.701843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.716643 (+ 14800us) server_negotiation.cc:407] Connection header received
0925 22:04:56.717337 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.717340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.717354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.717387 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.720167 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.720954 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.724539 (+ 3585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:56.724845 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:56.724905 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:56.735921 (+ 11016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:56.735926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:56.735927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:56.735941 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:56.738052 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.738597 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.738598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.738599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.738668 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:56.738882 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:56.738883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:56.738883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:56.739048 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:56.739137 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:56.739484 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 22:04:56.739584 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366}
I20260925 22:04:56.751766 32136 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:56.752846 32136 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:56.762535 32136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c7ca7ddfff141f1addfd3902ea0a52b: Generated new TSK 0
I20260925 22:04:56.762831 32136 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 22:04:56.789358 31937 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.787477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.9.112.126:40269)
0925 22:04:56.787537 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.787540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.788584 (+ 1044us) server_negotiation.cc:407] Connection header received
0925 22:04:56.788700 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.788702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.788728 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.788761 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.789268 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50402: BlockingRecv error: recv got EOF from 127.0.0.1:50402 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260925 22:04:56.790359 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.787909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.9.112.125:44907)
0925 22:04:56.787956 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.787958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.789598 (+ 1640us) server_negotiation.cc:407] Connection header received
0925 22:04:56.789850 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.789851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.789871 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.789901 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.790280 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58088: BlockingRecv error: recv got EOF from 127.0.0.1:58088 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 22:04:56.791020 32087 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.788253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.9.112.124:41263)
0925 22:04:56.788309 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.788311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.789797 (+ 1486us) server_negotiation.cc:407] Connection header received
0925 22:04:56.790587 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.790589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.790601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.790630 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.790946 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41042: BlockingRecv error: recv got EOF from 127.0.0.1:41042 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260925 22:04:56.793274 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.792537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.9.112.125:44907)
0925 22:04:56.792582 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.792584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.792818 (+ 234us) server_negotiation.cc:407] Connection header received
0925 22:04:56.792911 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.792912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.792929 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.792955 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.793226 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58090: BlockingRecv error: recv got EOF from 127.0.0.1:58090 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 22:04:56.796314 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.795504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.9.112.125:44907)
0925 22:04:56.795539 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.795541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.795812 (+ 271us) server_negotiation.cc:407] Connection header received
0925 22:04:56.795896 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.795897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.795908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.795933 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.796255 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58094: BlockingRecv error: recv got EOF from 127.0.0.1:58094 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 22:04:56.800504 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.799739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.9.112.125:44907)
0925 22:04:56.799774 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.799775 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.799985 (+ 210us) server_negotiation.cc:407] Connection header received
0925 22:04:56.800064 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.800065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.800075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.800100 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.800454 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58096: BlockingRecv error: recv got EOF from 127.0.0.1:58096 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 22:04:56.806669 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.805746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.9.112.125:44907)
0925 22:04:56.805788 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.805791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.806078 (+ 287us) server_negotiation.cc:407] Connection header received
0925 22:04:56.806188 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.806189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.806201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.806230 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.806609 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58098: BlockingRecv error: recv got EOF from 127.0.0.1:58098 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 22:04:56.817425 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.816262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.9.112.125:44907)
0925 22:04:56.816320 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.816322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.816715 (+ 393us) server_negotiation.cc:407] Connection header received
0925 22:04:56.816834 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.816835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.816849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.816882 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.817349 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58102: BlockingRecv error: recv got EOF from 127.0.0.1:58102 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 22:04:56.836620 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.835239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.9.112.125:44907)
0925 22:04:56.835307 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.835310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.835739 (+ 429us) server_negotiation.cc:407] Connection header received
0925 22:04:56.835924 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.835926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.835941 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.835975 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.836538 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58118: BlockingRecv error: recv got EOF from 127.0.0.1:58118 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 22:04:56.872233 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.871114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.9.112.125:44907)
0925 22:04:56.871198 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.871201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.871546 (+ 345us) server_negotiation.cc:407] Connection header received
0925 22:04:56.871636 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.871637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.871652 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.871727 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.872153 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58128: BlockingRecv error: recv got EOF from 127.0.0.1:58128 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 22:04:56.938364 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:56.937545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.9.112.125:44907)
0925 22:04:56.937667 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:56.937670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:56.937796 (+ 126us) server_negotiation.cc:407] Connection header received
0925 22:04:56.937870 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:56.937871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:56.937884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:56.937935 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:56.938310 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58140: BlockingRecv error: recv got EOF from 127.0.0.1:58140 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260925 22:04:57.068924 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:57.068150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.9.112.125:44907)
0925 22:04:57.068238 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.068240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.068253 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:04:57.068377 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.068379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.068395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.068449 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:57.068847 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58152: BlockingRecv error: recv got EOF from 127.0.0.1:58152 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260925 22:04:57.327641 32013 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:57.326778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.9.112.125:44907)
0925 22:04:57.326877 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.326880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.326900 (+ 20us) server_negotiation.cc:407] Connection header received
0925 22:04:57.326941 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.326943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.326955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.327053 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:57.327560 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58168: BlockingRecv error: recv got EOF from 127.0.0.1:58168 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 22:04:57.637739 32101 catalog_manager.cc:1259] Loaded cluster ID: 075efbd0bcbb404a94f99fb00922c28e
I20260925 22:04:57.637854 32101 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: loading cluster ID for follower catalog manager: success
I20260925 22:04:57.639142 32101 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: acquiring CA information for follower catalog manager: success
I20260925 22:04:57.639554 32101 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e0cc2abd30b45c6bf1c845cc54d1f78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:04:57.678391 32115 catalog_manager.cc:1259] Loaded cluster ID: 075efbd0bcbb404a94f99fb00922c28e
I20260925 22:04:57.678479 32115 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: loading cluster ID for follower catalog manager: success
I20260925 22:04:57.679507 32115 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: acquiring CA information for follower catalog manager: success
I20260925 22:04:57.680007 32115 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da7c6a0bce0a416c8c619e033e357fe9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:04:57.844587 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:57.841202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.9.112.125:44907)
0925 22:04:57.841375 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.841377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.841485 (+ 108us) server_negotiation.cc:407] Connection header received
0925 22:04:57.841543 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.841544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.841598 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.841704 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:57.841890 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.842525 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.843552 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.843783 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.843856 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:57.844025 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:57.844191 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:57.844262 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:57.844457 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 22:04:57.844482 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260925 22:04:57.849764 32219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:57.846964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.9.112.124:41263)
0925 22:04:57.847083 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.847084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.847158 (+ 74us) server_negotiation.cc:407] Connection header received
0925 22:04:57.847231 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.847232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.847268 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.847351 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:57.847482 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.847986 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.848946 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.849198 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.849261 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:57.849451 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:57.849566 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:57.849625 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:57.849661 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 22:04:57.849679 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260925 22:04:57.855082 32219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:57.851806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.9.112.124:41263)
0925 22:04:57.851861 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.851863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.852014 (+ 151us) server_negotiation.cc:407] Connection header received
0925 22:04:57.852127 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.852129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.852142 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.852198 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:57.852369 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.853152 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.854311 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.854615 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.854668 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:57.854787 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:57.854824 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:57.854865 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:57.854952 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 22:04:57.854985 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 22:04:57.855269 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:57.851755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.9.112.126:40269)
0925 22:04:57.851928 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.851931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.851996 (+ 65us) server_negotiation.cc:407] Connection header received
0925 22:04:57.852080 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.852082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.852140 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.852219 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:57.852429 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.853212 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.854132 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.854380 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.854445 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:57.854691 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:57.854824 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:57.854891 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:57.855140 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 22:04:57.855174 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260925 22:04:57.855254 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:57.852074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.9.112.125:44907)
0925 22:04:57.852125 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:57.852127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:57.852484 (+ 357us) server_negotiation.cc:407] Connection header received
0925 22:04:57.852539 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:57.852541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:57.852553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:57.852600 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:04:57.852718 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.853227 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.854355 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:57.854613 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:57.854680 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:57.854952 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:04:57.854998 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:04:57.855025 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:57.855131 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 22:04:57.855166 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:04:57.857525 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31866
I20260925 22:04:57.864028 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 31941
I20260925 22:04:57.869616 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 32016
2026-09-25T22:04:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2925 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 22:04:57 dist-test-slave-794d krb5kdc[32230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 22:04:57 dist-test-slave-794d krb5kdc[32230](info): set up 2 sockets
Sep 25 22:04:57 dist-test-slave-794d krb5kdc[32230](info): commencing operation
krb5kdc: starting...
W20260925 22:04:58.852855 9665 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/test-user.keytab.
Sep 25 22:04:58 dist-test-slave-794d krb5kdc[32230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T22:04:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T22:04:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.112.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
Entry for principal kudu/127.9.112.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab.
I20260925 22:04:58.994166 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40671
--webserver_interface=127.9.112.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40671,127.9.112.125:46367,127.9.112.124:38259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 22:04:59.087571 32243 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:04:59.088090 32243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:59.088135 32243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:59.088160 32243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:59.089545 32243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:59.089607 32243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:59.089620 32243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:59.089632 32243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:59.089644 32243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:59.091439 32243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40671,127.9.112.125:46367,127.9.112.124:38259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.126.keytab
--principal=kudu/127.9.112.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.126:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.112.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:59.091729 32243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:59.092005 32243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:59.097666 32248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:59.097602 32249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:59.097802 32251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:59.097965 32243 server_base.cc:1034] running on GCE node
I20260925 22:04:59.098443 32243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:59.098881 32243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:59.100059 32243 hybrid_clock.cc:648] HybridClock initialized: now 1790373899100027 us; error 52 us; skew 500 ppm
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:59.111304 32243 init.cc:377] Logged in from keytab as kudu/127.9.112.126@KRBTEST.COM (short username kudu)
I20260925 22:04:59.111871 32243 webserver.cc:460] Webserver started at http://127.9.112.126:39461/ using document root <none> and password file <none>
I20260925 22:04:59.112092 32243 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:59.112140 32243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:59.112247 32243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:59.113413 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data/instance:
uuid: "004c9edaec9141c2948a319ffbc3edcb"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "02fa21ea8d0605424bd9a5fc50110e18"
server_key_iv: "79a2a3c340534588341b0f0f17449a51"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.113722 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal/instance:
uuid: "004c9edaec9141c2948a319ffbc3edcb"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "02fa21ea8d0605424bd9a5fc50110e18"
server_key_iv: "79a2a3c340534588341b0f0f17449a51"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.115146 32243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 22:04:59.116050 32258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:59.116304 32243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 22:04:59.116487 32243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal
uuid: "004c9edaec9141c2948a319ffbc3edcb"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "02fa21ea8d0605424bd9a5fc50110e18"
server_key_iv: "79a2a3c340534588341b0f0f17449a51"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.116582 32243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:59.126643 32243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:59.129243 32243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:59.129371 32243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:59.133931 32243 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.126:40671
I20260925 22:04:59.133984 32310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.126:40671 every 8 connection(s)
I20260925 22:04:59.134354 32243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/data/info.pb
I20260925 22:04:59.135550 32311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:59.137584 32311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:59.140663 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 32243
I20260925 22:04:59.140786 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-0/wal/instance
I20260925 22:04:59.141017 9665 external_mini_cluster.cc:1428] Setting key 28d00bc0a72c2f6861f38fd67a3b2432
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.125@KRBTEST.COM; defaulting to no policy
I20260925 22:04:59.150571 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.138496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40671 (local address 127.0.0.1:48384)
0925 22:04:59.138704 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.138752 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.139360 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.139818 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.139821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.140113 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.140305 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.140309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.141980 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.141981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.142774 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.142778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.143661 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.143664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.143756 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.144426 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.144435 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.146265 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.148662 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.148673 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.148675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.148950 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.149301 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.149302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.149302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.149420 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.149932 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.149939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.150026 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.150139 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.150315 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260925 22:04:59.150578 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.138564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.9.112.126:40671)
0925 22:04:59.138792 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.138798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.138841 (+ 43us) server_negotiation.cc:407] Connection header received
0925 22:04:59.139600 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.139603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.139625 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.139749 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.140461 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.141863 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.142862 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.143398 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.143551 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.146382 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.146394 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.146396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.146405 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.148522 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.149096 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.149101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.149102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.149219 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.149488 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.149489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.149489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.149849 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.149942 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.150217 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.150391 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":179,"thread_start_us":117,"threads_started":1}
I20260925 22:04:59.151523 32311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:59.151965 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.151837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42516)
0925 22:04:59.151886 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.151935 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 22:04:59.152112 32259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
I20260925 22:04:59.152817 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.152690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42528)
0925 22:04:59.152753 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.152792 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.125@KRBTEST.COM" created.
W20260925 22:04:59.152953 32311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:46367: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab.
Entry for principal kudu/127.9.112.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab.
I20260925 22:04:59.173026 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:46367
--webserver_interface=127.9.112.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab
--principal=kudu/127.9.112.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40671,127.9.112.125:46367,127.9.112.124:38259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:04:59.202239 32311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } attempt: 1
I20260925 22:04:59.202966 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.202769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42544)
0925 22:04:59.202848 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.202921 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 22:04:59.203382 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.203269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42546)
0925 22:04:59.203322 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.203357 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 22:04:59.203583 32311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:46367: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
W20260925 22:04:59.269083 32318 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:04:59.269528 32318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:59.269625 32318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:59.269658 32318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 22:04:59.271826 32311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } attempt: 2
W20260925 22:04:59.272148 32318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:59.272244 32318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:59.272264 32318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:59.272282 32318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:59.272298 32318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:59.272607 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.272379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42550)
0925 22:04:59.272510 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.272573 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 22:04:59.273121 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.273027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42558)
0925 22:04:59.273076 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.273102 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 22:04:59.273226 32311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.125:46367: Network error: Client connection negotiation failed: client connection to 127.9.112.125:46367: connect: Connection refused (error 111)
I20260925 22:04:59.274725 32318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40671,127.9.112.125:46367,127.9.112.124:38259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.125.keytab
--principal=kudu/127.9.112.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.125:46367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.112.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:59.275084 32318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:59.275352 32318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:59.281286 32326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:59.281364 32323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:59.281440 32324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:59.281656 32318 server_base.cc:1034] running on GCE node
I20260925 22:04:59.281970 32318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:59.282358 32318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:59.283533 32318 hybrid_clock.cc:648] HybridClock initialized: now 1790373899283497 us; error 61 us; skew 500 ppm
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:59.286556 32318 init.cc:377] Logged in from keytab as kudu/127.9.112.125@KRBTEST.COM (short username kudu)
I20260925 22:04:59.287113 32318 webserver.cc:460] Webserver started at http://127.9.112.125:39619/ using document root <none> and password file <none>
I20260925 22:04:59.287357 32318 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:59.287408 32318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:59.287545 32318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:59.288830 32318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data/instance:
uuid: "2a120116cc994f07957cdcf05468f795"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "ce96de6b10bdc347c95af4ea0f503146"
server_key_iv: "593962a7fe2d1f788eca006b8970eea8"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.289234 32318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal/instance:
uuid: "2a120116cc994f07957cdcf05468f795"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "ce96de6b10bdc347c95af4ea0f503146"
server_key_iv: "593962a7fe2d1f788eca006b8970eea8"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.290782 32318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 22:04:59.291724 32333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:59.292001 32318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:59.292105 32318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal
uuid: "2a120116cc994f07957cdcf05468f795"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "ce96de6b10bdc347c95af4ea0f503146"
server_key_iv: "593962a7fe2d1f788eca006b8970eea8"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.292177 32318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:59.318331 32318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:59.321293 32318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:59.321478 32318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:59.327013 32318 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.125:46367
I20260925 22:04:59.327088 32385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.125:46367 every 8 connection(s)
I20260925 22:04:59.327530 32318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/data/info.pb
I20260925 22:04:59.328724 32386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:59.330037 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 32318
I20260925 22:04:59.330154 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-1/wal/instance
I20260925 22:04:59.330363 9665 external_mini_cluster.cc:1428] Setting key e4bcf4413a97e96de370dec0257a1b6c
I20260925 22:04:59.330528 32386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.112.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.112.124@KRBTEST.COM" created.
I20260925 22:04:59.342423 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.331328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.9.112.126:40671)
0925 22:04:59.331407 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.331409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.331694 (+ 285us) server_negotiation.cc:407] Connection header received
0925 22:04:59.332480 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.332482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.332500 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.332624 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.333554 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.334113 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.335755 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.335994 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.336067 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.338918 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.338920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.338921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.338931 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.340400 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.341055 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.341057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.341057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.341129 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.341401 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.341402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.341402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.341539 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.341604 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.342200 (+ 596us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.342321 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 22:04:59.342579 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.331266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40671 (local address 127.0.0.1:48396)
0925 22:04:59.331488 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.331560 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.332324 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.332760 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.332765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.333101 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.333425 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.333431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.334225 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.334226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.335633 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.335636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.336108 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.336109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.336292 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.336962 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.336975 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.338784 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.340496 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.340505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.340509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.340911 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.341207 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.341207 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.341208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.341329 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.341636 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.341643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.342022 (+ 379us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.342136 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.342331 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260925 22:04:59.344962 32386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:04:59.353020 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.345504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.9.112.125:46367)
0925 22:04:59.345729 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.345734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.345756 (+ 22us) server_negotiation.cc:407] Connection header received
0925 22:04:59.345774 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.345776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.345788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.345849 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.346167 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.346860 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.347511 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.347826 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.347886 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.350292 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.350307 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.350307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.350314 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.351841 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.352157 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.352160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.352160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.352207 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.352381 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.352381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.352382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.352548 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.352628 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.352744 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.352844 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":196,"thread_start_us":131,"threads_started":1}
I20260925 22:04:59.353034 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.345417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42564)
0925 22:04:59.345494 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.345539 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.345578 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.345861 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.345862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.346035 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.346097 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.346100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.346942 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.346943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.347443 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.347444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.347889 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.347890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.347963 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.348415 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.348421 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.350190 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.351930 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.351931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.351931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.352088 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.352260 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.352261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.352261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.352313 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.352611 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.352612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.352652 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.352713 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.352897 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":86}
I20260925 22:04:59.354049 32386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:59.354677 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.354403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37440)
0925 22:04:59.354463 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.354608 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 22:04:59.354878 32336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
I20260925 22:04:59.355413 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.355325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37442)
0925 22:04:59.355359 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.355386 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260925 22:04:59.355544 32386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38259: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.112.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab.
Entry for principal kudu/127.9.112.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab.
I20260925 22:04:59.361415 9665 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
/tmp/dist-test-taskQ0dh6G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:38259
--webserver_interface=127.9.112.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab
--principal=kudu/127.9.112.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.112.84:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.112.126:40671,127.9.112.125:46367,127.9.112.124:38259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 22:04:59.364445 32311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } attempt: 3
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:04:59.373458 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.364813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42578)
0925 22:04:59.364905 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.364926 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.365041 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.365272 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.365274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.365500 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.365605 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.365608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.366419 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.366420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.367320 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.367323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.367819 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.367821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.367913 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.368519 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.368526 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.370368 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.372234 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.372236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.372237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.372525 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.372730 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.372731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.372732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.372832 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.373090 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.373091 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.373155 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.373216 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.373342 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 22:04:59.373449 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.364895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.9.112.125:46367)
0925 22:04:59.364953 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.364955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.365120 (+ 165us) server_negotiation.cc:407] Connection header received
0925 22:04:59.365160 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.365161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.365173 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.365251 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.365735 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.366315 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.367440 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.367709 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.367785 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.370523 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.370526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.370526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.370541 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.372114 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.372600 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.372602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.372602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.372670 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.372894 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.372895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.372895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.373027 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.373092 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.373257 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.373344 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:04:59.374027 32311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } has no permanent_uuid. Determining permanent_uuid...
I20260925 22:04:59.374596 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.374357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37446)
0925 22:04:59.374513 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.374567 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260925 22:04:59.374984 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.374900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37450)
0925 22:04:59.374934 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.374963 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 22:04:59.375069 32311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38259: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
I20260925 22:04:59.404793 32386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } attempt: 1
I20260925 22:04:59.405433 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.405168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37456)
0925 22:04:59.405274 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.405367 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 22:04:59.405937 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.405826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37464)
0925 22:04:59.405879 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.405914 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 22:04:59.406054 32386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38259: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
I20260925 22:04:59.406181 32311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } attempt: 1
I20260925 22:04:59.406582 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.406396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37474)
0925 22:04:59.406466 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.406534 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:04:59.406939 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.406845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37488)
0925 22:04:59.406893 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.406920 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 22:04:59.407047 32311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38259: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
W20260925 22:04:59.458933 32393 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 22:04:59.459210 32393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 22:04:59.459227 32393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 22:04:59.459249 32393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 22:04:59.460446 32393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 22:04:59.460491 32393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 22:04:59.460503 32393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 22:04:59.460515 32393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 22:04:59.460525 32393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 22:04:59.462108 32393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.112.84:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.112.126:40671,127.9.112.125:46367,127.9.112.124:38259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/krb5kdc/kudu_127.9.112.124.keytab
--principal=kudu/127.9.112.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.112.124:38259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.112.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 20:55:39 UTC on ccycloud.quasar-hwtoeh.root.comops.site
build id 4673252
I20260925 22:04:59.462370 32393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 22:04:59.462599 32393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 22:04:59.468221 32399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:59.468214 32401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 22:04:59.468230 32398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 22:04:59.468827 32393 server_base.cc:1034] running on GCE node
I20260925 22:04:59.469152 32393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 22:04:59.469555 32393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 22:04:59.470763 32393 hybrid_clock.cc:648] HybridClock initialized: now 1790373899470741 us; error 38 us; skew 500 ppm
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 22:04:59.473799 32393 init.cc:377] Logged in from keytab as kudu/127.9.112.124@KRBTEST.COM (short username kudu)
I20260925 22:04:59.474181 32386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } attempt: 2
I20260925 22:04:59.474334 32393 webserver.cc:460] Webserver started at http://127.9.112.124:37231/ using document root <none> and password file <none>
I20260925 22:04:59.474663 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.474535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37492)
0925 22:04:59.474597 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.474631 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 22:04:59.474737 32393 fs_manager.cc:359] Metadata directory not provided
I20260925 22:04:59.474874 32393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 22:04:59.474974 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.474895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37508)
0925 22:04:59.474933 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.474956 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 22:04:59.475060 32386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38259: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
I20260925 22:04:59.475075 32393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 22:04:59.476495 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data/instance:
uuid: "1be0ee471ac94c339ec1af89999dd5cb"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "1c3afeb5cd19a24d24b727573698e5b8"
server_key_iv: "b1fce610a86b4002e6dd653524b6f656"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.476943 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal/instance:
uuid: "1be0ee471ac94c339ec1af89999dd5cb"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "1c3afeb5cd19a24d24b727573698e5b8"
server_key_iv: "b1fce610a86b4002e6dd653524b6f656"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.478663 32393 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 22:04:59.479651 32408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 22:04:59.480156 32393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 22:04:59.480268 32393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal
uuid: "1be0ee471ac94c339ec1af89999dd5cb"
format_stamp: "Formatted at 2026-09-25 22:04:59 on dist-test-slave-794d"
server_key: "1c3afeb5cd19a24d24b727573698e5b8"
server_key_iv: "b1fce610a86b4002e6dd653524b6f656"
server_key_version: "encryptionkey@0"
I20260925 22:04:59.480376 32393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 22:04:59.482235 32311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } attempt: 2
I20260925 22:04:59.482806 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.482672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37518)
0925 22:04:59.482738 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.482775 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 22:04:59.483355 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.483151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37526)
0925 22:04:59.483222 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.483331 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 22:04:59.483512 32311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.112.124:38259: Network error: Client connection negotiation failed: client connection to 127.9.112.124:38259: connect: Connection refused (error 111)
I20260925 22:04:59.490794 32393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 22:04:59.493024 32393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 22:04:59.493172 32393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 22:04:59.497570 32393 rpc_server.cc:307] RPC server started. Bound to: 127.9.112.124:38259
I20260925 22:04:59.497632 32460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.112.124:38259 every 8 connection(s)
I20260925 22:04:59.498005 32393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/data/info.pb
I20260925 22:04:59.498293 9665 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu as pid 32393
I20260925 22:04:59.498435 9665 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ0dh6G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790373628768857-9665-0/minicluster-data/master-2/wal/instance
I20260925 22:04:59.498704 9665 external_mini_cluster.cc:1428] Setting key 3610d49fe73388670e9d0d7d1cb2cf92
I20260925 22:04:59.499466 32461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 22:04:59.501885 32461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:59.511514 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.502666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.9.112.126:40671)
0925 22:04:59.502748 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.502751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.502948 (+ 197us) server_negotiation.cc:407] Connection header received
0925 22:04:59.503060 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.503062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.503075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.503145 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.503702 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.504401 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.505594 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.505797 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.505871 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.508619 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.508620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.508620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.508629 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.510063 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.510458 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.510459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.510459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.510525 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.510813 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.510814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.510815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.510947 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.511031 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.511312 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.511406 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 22:04:59.511737 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.502591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.126:40671 (local address 127.0.0.1:48402)
0925 22:04:59.502789 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.502838 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.502942 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.503165 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.503168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.503514 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.503600 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.503602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.504483 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.504485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.505475 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.505478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.505935 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.505936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.506025 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.506637 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.506647 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.508501 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.510140 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.510147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.510151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.510388 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.510607 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.510607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.510608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.510742 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.511043 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.511049 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.511154 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.511280 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.511487 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260925 22:04:59.512006 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.499377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.9.112.124:38259)
0925 22:04:59.499593 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.499604 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.499641 (+ 37us) server_negotiation.cc:407] Connection header received
0925 22:04:59.500120 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.500127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.500187 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.500327 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.501307 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.502350 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.503447 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.504068 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.504185 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.507361 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.507371 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.507371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.507380 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.509512 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.509925 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.509930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.509930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.510054 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.510601 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.510602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.510602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.510965 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.511175 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.511363 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.511882 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260925 22:04:59.512669 32461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.126@KRBTEST.COM
I20260925 22:04:59.521713 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.513010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.125:46367 (local address 127.0.0.1:42592)
0925 22:04:59.513154 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.513168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.513230 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.513387 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.513387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.513557 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.513642 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.513645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.514502 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.514503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.515387 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.515390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.516069 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.516070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.516157 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.516777 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.516783 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.518390 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.520495 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.520497 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.520497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.520731 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.521008 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.521009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.521009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.521072 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.521355 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.521356 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.521406 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.521478 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.521602 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260925 22:04:59.521729 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.513101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.9.112.125:46367)
0925 22:04:59.513174 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.513177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.513221 (+ 44us) server_negotiation.cc:407] Connection header received
0925 22:04:59.513292 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.513293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.513305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.513387 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.513730 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.514392 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.515512 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.515809 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.515905 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.518520 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.518523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.518523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.518533 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.520408 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.520843 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.520845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.520846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.520934 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.521137 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.521138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.521138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.521289 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.521358 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.521518 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.521620 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 22:04:59.521772 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.513120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.9.112.126:40671)
0925 22:04:59.513173 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.513176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.513222 (+ 46us) server_negotiation.cc:407] Connection header received
0925 22:04:59.513291 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.513292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.513304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.513379 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.514064 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.514656 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.515655 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.515918 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.515986 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.519094 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.519096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.519096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.519105 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.520655 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.521034 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.521035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.521035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.521083 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.521263 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.521264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.521264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.521410 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.521574 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.521616 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.521689 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 22:04:59.522791 32461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.124@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.112.125@KRBTEST.COM
I20260925 22:04:59.531448 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.523246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37544 (local address 127.9.112.124:38259)
0925 22:04:59.523689 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.523691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.523731 (+ 40us) server_negotiation.cc:407] Connection header received
0925 22:04:59.523802 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.523803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.523813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.523859 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.524157 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.524816 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.525535 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.525756 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.525815 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.528014 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.528016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.528017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.528072 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.529894 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.530423 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.530424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.530424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.530512 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.530820 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.530821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.530821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.530967 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.531050 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.531208 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.531315 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":422}
I20260925 22:04:59.531510 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.523153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37544)
0925 22:04:59.523670 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.523685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.523751 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.523861 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.523862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.524038 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.524084 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.524086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.524912 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.524913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.525447 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.525448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.525821 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.525822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.525866 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.526336 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.526342 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.527885 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.530013 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.530015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.530015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.530262 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.530623 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.530624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.530625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.530706 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.531066 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.531067 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.531110 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.531177 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.531385 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":97}
I20260925 22:04:59.532670 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.523141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.9.112.125:46367)
0925 22:04:59.523205 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.523207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.523216 (+ 9us) server_negotiation.cc:407] Connection header received
0925 22:04:59.523256 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.523257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.523269 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.523326 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.523869 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.524679 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.525666 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.525951 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.526136 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.529345 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.529347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.529348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.529358 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.531179 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.531522 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.531523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.531524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.531586 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.531877 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.531878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.531878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.532042 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.532183 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.532384 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.532476 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 22:04:59.533860 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: Bootstrap starting.
I20260925 22:04:59.534966 32461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:59.534988 9665 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 22:04:59.535319 32461 log.cc:824] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:59.536571 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: No bootstrap required, opened a new log
I20260925 22:04:59.536626 32285 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "004c9edaec9141c2948a319ffbc3edcb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48408
I20260925 22:04:59.538653 32461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.538821 32461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be0ee471ac94c339ec1af89999dd5cb, State: Initialized, Role: FOLLOWER
I20260925 22:04:59.538913 32461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.539325 32461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:59.539304 32467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.539479 32467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: This master's current role is: FOLLOWER
W20260925 22:04:59.541633 32478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:59.541718 32478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:59.542166 32393 master_runner.cc:186] Error getting master registration for 127.9.112.126:40671: OK, instance_id { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" instance_seqno: 1790373899132458 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:04:59.545475 32393 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:04:59.566251 32386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } attempt: 3
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.125@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
I20260925 22:04:59.575214 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.566677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37554)
0925 22:04:59.566747 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.566758 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.566804 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.567167 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.567169 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.567325 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.567381 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.567383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.568449 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.568450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.569379 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.569383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.569939 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.569940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.570012 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.570616 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.570621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.571935 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.573965 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.573966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.573966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.574169 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.574387 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.574387 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.574388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.574452 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.574849 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.574850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.574907 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.574983 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.575094 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 22:04:59.575244 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.566754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.9.112.124:38259)
0925 22:04:59.566856 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.566860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.566880 (+ 20us) server_negotiation.cc:407] Connection header received
0925 22:04:59.566921 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.566922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.566939 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.567052 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.567468 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.568299 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.569508 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.569847 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.569937 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.572026 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.572027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.572028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.572039 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.573875 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.574244 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.574245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.574245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.574315 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.574520 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.574521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.574521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.574775 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.574852 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.575025 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.575119 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 22:04:59.577790 32386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: Bootstrap starting.
I20260925 22:04:59.578485 32386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:59.578774 32386 log.cc:824] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:59.579885 32386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: No bootstrap required, opened a new log
I20260925 22:04:59.582430 32386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.582666 32386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a120116cc994f07957cdcf05468f795, State: Initialized, Role: FOLLOWER
I20260925 22:04:59.582774 32386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.582712 32311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } attempt: 3
I20260925 22:04:59.583319 32386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:59.583313 32481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.583557 32481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: This master's current role is: FOLLOWER
Sep 25 22:04:59 dist-test-slave-794d krb5kdc[32230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790373899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.112.126@KRBTEST.COM for kudu/127.9.112.124@KRBTEST.COM
I20260925 22:04:59.586216 32318 master_runner.cc:186] Error getting master registration for 127.9.112.126:40671: OK, instance_id { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" instance_seqno: 1790373899132458 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 22:04:59.589515 32318 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:04:59.591545 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.583518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.9.112.124:38259)
0925 22:04:59.583751 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.583753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.583764 (+ 11us) server_negotiation.cc:407] Connection header received
0925 22:04:59.583796 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.583797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.583809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.583886 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.584268 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.585065 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.585998 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.586248 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.586317 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.588902 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.588904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.588904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.588913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.590450 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.590798 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.590799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.590799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.590853 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.591016 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.591017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.591017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.591145 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.591202 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.591367 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.591443 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260925 22:04:59.591554 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.583203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.112.124:38259 (local address 127.0.0.1:37558)
0925 22:04:59.583344 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 22:04:59.583356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 22:04:59.583408 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 22:04:59.583907 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 22:04:59.583908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 22:04:59.584121 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 22:04:59.584184 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.584187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.585154 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.585155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.585768 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 22:04:59.585770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.586337 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.586339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 22:04:59.586486 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.587018 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 22:04:59.587024 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 22:04:59.588798 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 22:04:59.590539 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.590541 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.590542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.590727 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.590907 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 22:04:59.590908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 22:04:59.590908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 22:04:59.590964 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 22:04:59.591200 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 22:04:59.591201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 22:04:59.591270 (+ 69us) client_negotiation.cc:769] Sending connection context
0925 22:04:59.591332 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 22:04:59.591443 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
W20260925 22:04:59.593355 32492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 22:04:59.593456 32492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 22:04:59.593490 32311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb: Bootstrap starting.
I20260925 22:04:59.594310 32311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb: Neither blocks nor log segments found. Creating new log.
I20260925 22:04:59.594585 32311 log.cc:824] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb: Log is configured to *not* fsync() on all Append() calls
I20260925 22:04:59.595449 32311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb: No bootstrap required, opened a new log
I20260925 22:04:59.596949 32311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.597118 32311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004c9edaec9141c2948a319ffbc3edcb, State: Initialized, Role: FOLLOWER
I20260925 22:04:59.597213 32311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.597573 32311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 22:04:59.597604 32285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 22:04:59.597648 32285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:59.598527 32285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.598809 32285 leader_election.cc:288] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [CANDIDATE]: Term 1 election: Requested vote from peers 2a120116cc994f07957cdcf05468f795 (127.9.112.125:46367), 1be0ee471ac94c339ec1af89999dd5cb (127.9.112.124:38259)
I20260925 22:04:59.599318 32360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "004c9edaec9141c2948a319ffbc3edcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2a120116cc994f07957cdcf05468f795"
I20260925 22:04:59.599498 32360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:59.600502 32360 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 004c9edaec9141c2948a319ffbc3edcb in term 1.
I20260925 22:04:59.600564 32435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "004c9edaec9141c2948a319ffbc3edcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1be0ee471ac94c339ec1af89999dd5cb"
I20260925 22:04:59.600741 32435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [term 0 FOLLOWER]: Advancing to term 1
I20260925 22:04:59.601466 32435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 004c9edaec9141c2948a319ffbc3edcb in term 1.
I20260925 22:04:59.601745 32261 leader_election.cc:302] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c9edaec9141c2948a319ffbc3edcb, 1be0ee471ac94c339ec1af89999dd5cb; no voters:
I20260925 22:04:59.601997 32507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 22:04:59.602164 32507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [term 1 LEADER]: Becoming Leader. State: Replica: 004c9edaec9141c2948a319ffbc3edcb, State: Running, Role: LEADER
I20260925 22:04:59.602272 32507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } }
I20260925 22:04:59.603281 32507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 004c9edaec9141c2948a319ffbc3edcb. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.603377 32507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:59.603601 32514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 22:04:59.597534 32495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.604024 32495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:59.604779 32243 master_runner.cc:263] No leader master found from master 004c9edaec9141c2948a319ffbc3edcb
I20260925 22:04:59.604830 32243 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 22:04:59.605104 32514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 22:04:59.606609 32435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [term 1 FOLLOWER]: Refusing update from remote peer 004c9edaec9141c2948a319ffbc3edcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:59.606608 32360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [term 1 FOLLOWER]: Refusing update from remote peer 004c9edaec9141c2948a319ffbc3edcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 22:04:59.606910 32495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:59.607147 32495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 22:04:59.608397 32481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 004c9edaec9141c2948a319ffbc3edcb. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.608510 32481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:59.608330 32467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 004c9edaec9141c2948a319ffbc3edcb. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.608621 32467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:59.609781 32495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.609861 32495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:59.610332 32481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.610425 32481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:59.610682 32517 mvcc.cc:205] Tried to move back new op lower bound from 7333371492785999872 to 7333371492773953536. Current Snapshot: MvccSnapshot[applied={T|T < 7333371492785999872}]
I20260925 22:04:59.611248 32519 mvcc.cc:205] Tried to move back new op lower bound from 7333371492785999872 to 7333371492773953536. Current Snapshot: MvccSnapshot[applied={T|T < 7333371492785999872}]
I20260925 22:04:59.612793 32514 catalog_manager.cc:1347] Generated new cluster ID: 528ef5e30eb3418488b35d2ea8f6f1bb
I20260925 22:04:59.612856 32514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 22:04:59.624981 32507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.625130 32507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb [sys.catalog]: This master's current role is: LEADER
I20260925 22:04:59.625756 32467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "004c9edaec9141c2948a319ffbc3edcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "004c9edaec9141c2948a319ffbc3edcb" member_type: VOTER last_known_addr { host: "127.9.112.126" port: 40671 } } peers { permanent_uuid: "2a120116cc994f07957cdcf05468f795" member_type: VOTER last_known_addr { host: "127.9.112.125" port: 46367 } } peers { permanent_uuid: "1be0ee471ac94c339ec1af89999dd5cb" member_type: VOTER last_known_addr { host: "127.9.112.124" port: 38259 } } }
I20260925 22:04:59.625876 32467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb [sys.catalog]: This master's current role is: FOLLOWER
I20260925 22:04:59.626488 32518 mvcc.cc:205] Tried to move back new op lower bound from 7333371492785999872 to 7333371492773953536. Current Snapshot: MvccSnapshot[applied={T|T < 7333371492785999872}]
I20260925 22:04:59.627166 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.608925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.9.112.126:40671)
0925 22:04:59.608982 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.608984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.612521 (+ 3537us) server_negotiation.cc:407] Connection header received
0925 22:04:59.612631 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.612633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.612645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.612686 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.620448 (+ 7762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.621120 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.622048 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.622326 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.622361 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.624180 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.624181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.624181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.624189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.625886 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.626273 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.626274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.626275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.626329 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.626511 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.626512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.626512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.626647 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.626752 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.626963 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.627053 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":23}
I20260925 22:04:59.628417 32514 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 22:04:59.629240 32514 catalog_manager.cc:1504] Loading token signing keys...
I20260925 22:04:59.630579 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.604323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.9.112.125:46367)
0925 22:04:59.604410 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.604413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.610017 (+ 5604us) server_negotiation.cc:407] Connection header received
0925 22:04:59.610134 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.610136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.610150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.610191 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.616081 (+ 5890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.616914 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.618020 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.618344 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.618389 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.627231 (+ 8842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.627233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.627234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.627244 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.629188 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.629688 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.629689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.629690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.629750 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.629935 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.629936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.629936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.630068 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.630169 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.630392 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.630481 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 22:04:59.638904 32514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 004c9edaec9141c2948a319ffbc3edcb: Generated new TSK 0
I20260925 22:04:59.639329 32514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 22:04:59.643637 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:04:59.609393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.9.112.124:38259)
0925 22:04:59.609441 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.609443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.618916 (+ 9473us) server_negotiation.cc:407] Connection header received
0925 22:04:59.619035 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.619037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.619051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.619081 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.619883 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.620640 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.621985 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:04:59.622283 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:04:59.622323 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:04:59.639938 (+ 17615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 22:04:59.639940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 22:04:59.639940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 22:04:59.639953 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 22:04:59.641947 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.642417 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.642418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.642419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.642473 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 22:04:59.642648 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 22:04:59.642648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 22:04:59.642649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 22:04:59.642822 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 22:04:59.642916 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 22:04:59.643440 (+ 524us) server_negotiation.cc:299] Negotiation successful
0925 22:04:59.643520 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 22:04:59.673439 32314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.671978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.9.112.126:40671)
0925 22:04:59.672029 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.672031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.672728 (+ 697us) server_negotiation.cc:407] Connection header received
0925 22:04:59.672837 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.672838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.672849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.672879 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.673340 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48418: BlockingRecv error: recv got EOF from 127.0.0.1:48418 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 22:04:59.674715 32464 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.672428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.9.112.124:38259)
0925 22:04:59.672472 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.672474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.674128 (+ 1654us) server_negotiation.cc:407] Connection header received
0925 22:04:59.674269 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.674270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.674283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.674312 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.674646 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37574: BlockingRecv error: recv got EOF from 127.0.0.1:37574 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 22:04:59.675741 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.673939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.9.112.125:46367)
0925 22:04:59.673984 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.673985 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.675067 (+ 1082us) server_negotiation.cc:407] Connection header received
0925 22:04:59.675158 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.675160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.675170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.675250 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.675678 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42630: BlockingRecv error: recv got EOF from 127.0.0.1:42630 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 22:04:59.685518 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.683964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.9.112.125:46367)
0925 22:04:59.684028 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.684044 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.684695 (+ 651us) server_negotiation.cc:407] Connection header received
0925 22:04:59.684847 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.684849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.684876 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.684911 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.685447 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42640: BlockingRecv error: recv got EOF from 127.0.0.1:42640 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260925 22:04:59.691787 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.690847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.9.112.125:46367)
0925 22:04:59.690894 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.690896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.691169 (+ 273us) server_negotiation.cc:407] Connection header received
0925 22:04:59.691272 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.691274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.691285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.691313 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.691725 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42642: BlockingRecv error: recv got EOF from 127.0.0.1:42642 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:04:59.701247 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.696642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.9.112.125:46367)
0925 22:04:59.696686 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.696688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.700607 (+ 3919us) server_negotiation.cc:407] Connection header received
0925 22:04:59.700738 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.700740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.700755 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.700784 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.701184 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42654: BlockingRecv error: recv got EOF from 127.0.0.1:42654 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 22:04:59.707564 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.706689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.9.112.125:46367)
0925 22:04:59.706731 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.706733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.706982 (+ 249us) server_negotiation.cc:407] Connection header received
0925 22:04:59.707080 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.707081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.707093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.707121 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.707490 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42670: BlockingRecv error: recv got EOF from 127.0.0.1:42670 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 22:04:59.718151 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.717176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.9.112.125:46367)
0925 22:04:59.717223 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.717225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.717504 (+ 279us) server_negotiation.cc:407] Connection header received
0925 22:04:59.717633 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.717635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.717648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.717679 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.718085 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42682: BlockingRecv error: recv got EOF from 127.0.0.1:42682 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:04:59.736685 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.735675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.9.112.125:46367)
0925 22:04:59.735721 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.735723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.735993 (+ 270us) server_negotiation.cc:407] Connection header received
0925 22:04:59.736101 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.736103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.736116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.736145 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.736613 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42684: BlockingRecv error: recv got EOF from 127.0.0.1:42684 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 22:04:59.771425 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.770429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.9.112.125:46367)
0925 22:04:59.770480 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.770482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.770704 (+ 222us) server_negotiation.cc:407] Connection header received
0925 22:04:59.770799 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.770800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.770818 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.770898 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.771344 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42696: BlockingRecv error: recv got EOF from 127.0.0.1:42696 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 22:04:59.837639 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.836873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.9.112.125:46367)
0925 22:04:59.836972 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.836975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.836991 (+ 16us) server_negotiation.cc:407] Connection header received
0925 22:04:59.837030 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.837031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.837044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.837144 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.837571 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42698: BlockingRecv error: recv got EOF from 127.0.0.1:42698 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260925 22:04:59.968053 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:04:59.967171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42708 (local address 127.9.112.125:46367)
0925 22:04:59.967296 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 22:04:59.967299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:04:59.967390 (+ 91us) server_negotiation.cc:407] Connection header received
0925 22:04:59.967455 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:04:59.967456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:04:59.967470 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:04:59.967533 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:04:59.967977 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42708: BlockingRecv error: recv got EOF from 127.0.0.1:42708 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 22:05:00.226353 32391 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 22:05:00.225633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.9.112.125:46367)
0925 22:05:00.225699 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:00.225702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:00.225715 (+ 13us) server_negotiation.cc:407] Connection header received
0925 22:05:00.225829 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:00.225831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:00.225844 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:00.225903 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 22:05:00.226269 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42710: BlockingRecv error: recv got EOF from 127.0.0.1:42710 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 22:05:00.542213 32478 catalog_manager.cc:1259] Loaded cluster ID: 528ef5e30eb3418488b35d2ea8f6f1bb
I20260925 22:05:00.542297 32478 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: loading cluster ID for follower catalog manager: success
I20260925 22:05:00.543087 32478 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: acquiring CA information for follower catalog manager: success
I20260925 22:05:00.543332 32478 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1be0ee471ac94c339ec1af89999dd5cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:05:00.593955 32492 catalog_manager.cc:1259] Loaded cluster ID: 528ef5e30eb3418488b35d2ea8f6f1bb
I20260925 22:05:00.594051 32492 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: loading cluster ID for follower catalog manager: success
I20260925 22:05:00.594831 32492 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: acquiring CA information for follower catalog manager: success
I20260925 22:05:00.595136 32492 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a120116cc994f07957cdcf05468f795: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 22:05:00.742918 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:00.739808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.9.112.125:46367)
0925 22:05:00.739973 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:00.739976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:00.740038 (+ 62us) server_negotiation.cc:407] Connection header received
0925 22:05:00.740096 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:00.740098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:00.740185 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:00.740309 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:00.740567 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.741167 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.742090 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.742299 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.742357 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:00.742530 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:00.742672 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:00.742747 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:00.742767 (+ 20us) server_negotiation.cc:299] Negotiation successful
0925 22:05:00.742795 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":110,"threads_started":1}
I20260925 22:05:00.747637 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:00.744953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.9.112.124:38259)
0925 22:05:00.745130 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:00.745133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:00.745168 (+ 35us) server_negotiation.cc:407] Connection header received
0925 22:05:00.745208 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:00.745209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:00.745246 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:00.745314 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:00.745436 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.745924 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.746784 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.746987 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.747062 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:00.747261 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:00.747376 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:00.747443 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:00.747532 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 22:05:00.747553 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
I20260925 22:05:00.752862 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:00.749906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.9.112.124:38259)
0925 22:05:00.749977 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:00.749979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:00.750115 (+ 136us) server_negotiation.cc:407] Connection header received
0925 22:05:00.750218 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:00.750219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:00.750229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:00.750274 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:00.750414 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.750918 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.751987 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.752289 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.752379 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:00.752531 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:00.752567 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:00.752613 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:00.752723 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 22:05:00.752766 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 22:05:00.752878 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:00.749625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.9.112.125:46367)
0925 22:05:00.749675 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:00.749678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:00.749958 (+ 280us) server_negotiation.cc:407] Connection header received
0925 22:05:00.750073 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:00.750075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:00.750091 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:00.750141 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:00.750358 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.751150 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.752074 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.752395 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.752454 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:00.752606 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:00.752654 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:00.752696 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:00.752778 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 22:05:00.752806 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 22:05:00.753312 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 22:05:00.749596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.9.112.126:40671)
0925 22:05:00.749740 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 22:05:00.749742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 22:05:00.749804 (+ 62us) server_negotiation.cc:407] Connection header received
0925 22:05:00.749914 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 22:05:00.749915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 22:05:00.749984 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 22:05:00.750095 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 22:05:00.750314 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.751148 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.752192 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 22:05:00.752498 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 22:05:00.752742 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 22:05:00.752828 (+ 86us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 22:05:00.752949 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 22:05:00.753179 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 22:05:00.753195 (+ 16us) server_negotiation.cc:299] Negotiation successful
0925 22:05:00.753220 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260925 22:05:00.755290 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 32243
I20260925 22:05:00.761716 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 32318
I20260925 22:05:00.766973 9665 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ0dh6G/build/release/bin/kudu with pid 32393
2026-09-25T22:05:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2897 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5823 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (272006 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST